scie.nti.st

Using duplicity for full server backup on Ubuntu 12.04

2013-04-13T05:23:00-07:00

duplicity is a great full featured backup tool, providing "encrypted bandwidth-efficient backups using the rsync algorithm." I’m choosing it for my backup needs primarily for the follow reasons:

Supports a fleet of backup server types, and even a chroot’d SFTP server will work
Works with Linux, FreeBSD, and OpenBSD, out of the box
Does not require root access on the backup server

To perform full server backups using duplicity on your Ubuntu 12.04 system, simply perform the following.

Install duplicity:
```
sudo apt-get install duplicity
```
Perform backup (using rsync method):
```
$ sudo -H duplicity --no-encryption \
  --exclude /mnt --exclude /tmp --exclude /proc --exclude /sys \
  / rsync://<user>@<host>//<backup-directory>
```
You’ll need to replace <user>, <host> and <backup-directory> with your backup username, host, and directory, respectively. See the "Full Example" below for more info.

Output should look something like:
```
Import of duplicity.backends.sshbackend Failed: No module named paramiko
Import of duplicity.backends.giobackend Failed: No module named gio
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: none
No signatures found, switching to full backup.
```
If you’d like your backups to be encrypted, simply remove the –no-encryption option and provide a passphrase when prompted. You’ll need this same passphrase to restore from a backup. It is a symmetric key. Keep the key very safe; if you lose it, your backups will be useless.

Schedule regular backups

The next step is to set this up in cron, so your backups are done regularly. The following entries in /etc/crontab will perform a full backup once a week, Sunday at midnight, then incremental backups the other 6 days of the week (also at midnight). Additionally, we want to clean up backups that are too old, so we add an additional entry to keep only the latest two full backups (plus incrementals).

0 0 * * sun     root    duplicity full --no-encryption --exclude /mnt --exclude /tmp --exclude /proc --exclude /sys / rsync://<user>@<host>//<backup-directory>
0 0 * * 1-6     root    duplicity --no-encryption --exclude /mnt --exclude /tmp --exclude /proc --exclude /sys / rsync://<user>@<host>//<backup-directory>
0 8 * * sun     root    duplicity remove-all-but-n-full 2 --force rsync://<user>@<host>//<backup-directory>

Full Example for an ARP Networks VPS

At ARP Networks, we provide raw backup space to customers for 10 cents per GB. The example below will illustrate how to do a full backup of your Ubuntu or Debian VPS. Assuming your backup username is johndoe, simply perform the following steps.

Add root SSH public key to backup server

Create an SSH key for root as follows:
```
sudo -H ssh-keygen -t rsa -b 2048
```
Accept the defaults and choose a good passphrase.

See the Portal, click on the Backup Space service, and then click the link to submit an SSH key.
Install duplicity:
```
sudo apt-get install duplicity
```
Perform backup (using rsync method):
```
$ sudo -H duplicity --no-encryption \
  --exclude /mnt --exclude /tmp --exclude /proc --exclude /sys \
  / rsync://johndoe@backup01.cust.arpnetworks.com//home/johndoe
```
backup01.cust.arpnetworks.com is one of our backup servers. If the Portal lists a different server for you, use that one.

Additionally, as mentioned above, if you want your backups to be encrypted, remove the –no-encryption option.

Schedule regular backups

Put the following in your /etc/crontab

0 0 * * sun     root    duplicity full --no-encryption --exclude /mnt --exclude /tmp --exclude /proc --exclude /sys / rsync://johndoe@backup01.cust.arpnetworks.com//home/johndoe
0 0 * * 1-6     root    duplicity --no-encryption --exclude /mnt --exclude /tmp --exclude /proc --exclude /sys / rsync://johndoe@backup01.cust.arpnetworks.com//home/johndoe
0 8 * * sun     root    duplicity remove-all-but-n-full 2 --force rsync://johndoe@backup01.cust.arpnetworks.com//home/johndoe

Note: Please adjust the backup time to your own preference, otherwise many backups may fire off at the exact same time, slowing down the host.

…and that’s all there is to it!

How to add a new partition on OpenBSD

2013-03-05T03:30:00-08:00

Following the same theme as my last post, instead of claiming space added to the end of a volume by extending an existing partition (technically, a "slice" in OpenBSD), I will show how to add a new partition using this space.

I find it convenient to create the new slice as the "d" slice and then put /home onto it.

So, let’s get started.

Boot into the system’s rescue disk:
```
> boot bsd.rd
```
Then select (S)hell at the prompt.

Claim extra space

Re-initialize the MBR, using the entire disk.

# fdisk -i wd0
Do you wish to write new MBR and partition table? [n] y
Writing MBR at offset 0.
#

Add a new slice for /home (the "d" slice)

This system has only the root slice and swap. We will add a new slice, using all the unused space at the end. In the following example, I have a 120GB volume, where 20GB is used by root and swap, and the rest unallocated.

# disklabel -E wd0
Label editor (enter '?' for help at any prompt)
> p
OpenBSD area: 64-251658225; size: 251658161; free: 209728575
#                size           offset  fstype [fsize bsize  cpg]
  a:         38812976               64  4.2BSD   2048 16384 38128
  b:          3116610         38813040    swap
  c:        251658240                0  unused
> a
partition: [d]
offset: [41929650]
size: [209728575]
FS type: [4.2BSD]
Rounding offset to bsize (32 sectors): 41929664
Rounding size to bsize (32 sectors): 209728544
> p
OpenBSD area: 64-251658225; size: 251658161; free: 31
#                size           offset  fstype [fsize bsize  cpg]
  a:         38812976               64  4.2BSD   2048 16384 38128
  b:          3116610         38813040    swap
  c:        251658240                0  unused
  d:        209728544         41929664  4.2BSD   2048 16384    1
> w
> q
No label changes.
#

Create a filesystem on the new slice:

# newfs /dev/rwd0d
/dev/rwd0d: 102406.5MB in 209728544 sectors of 512 bytes
506 cylinder groups of 202.47MB, 12958 blocks, 25984 inodes each
super-block backups (for fsck -b #) at:
 32, 414688, 829344, 1244000, 1658656, 2073312, 2487968, 2902624, 3317280,
 3731936, 4146592, 4561248, 4975904, 5390560, 5805216, 6219872, 6634528,
 7049184, 7463840, 7878496, 8293152, 8707808, 9122464, 9537120, 9951776,
 ...
 [snip]
 ...
 193229728, 193644384, 194059040, 194473696, 194888352, 195303008, 195717664,
 196132320, 196546976, 196961632, 197376288, 197790944, 198205600, 198620256,
 199034912, 199449568, 199864224, 200278880, 200693536, 201108192, 201522848,
 201937504, 202352160, 202766816, 203181472, 203596128, 204010784, 204425440,
 204840096, 205254752, 205669408, 206084064, 206498720, 206913376, 207328032,
 207742688, 208157344, 208572000, 208986656, 209401312,
#

Migrate /home

Here, we mount the root filesystem (which contains the old /home), and also the new filesystem, and finally migrate the data from old /home to new space.
```
# mount /dev/wd0a /mnt
# mount /dev/wd0d /mnt2
# (cd /mnt/home; tar cf - .) | (cd /mnt2; tar xpf -)
# rm -rf /mnt/home
# mkdir /mnt/home
```
Note, the tar command above is a nice platform independent way of copying everything from one directory to another, preserving everything (permissions, etc…). The cp command is different on Linux and the *BSDs, yet tar, used in the above fasion, is identical on Linux, FreeBSD, NetBSD, and OpenBSD. I thank Todd Fries of Free Daemon Consulting for teaching me that one.
Mount on boot

The following command will add the new /home to your /etc/fstab so it is automatically mounted upon boot.
```
# echo "/dev/wd0d /home ffs rw,softdep 0 1" >> /mnt/etc/fstab
```
Clean up:
```
# umount /mnt*
# reboot
```
Now enjoy the extra space!

How to resize an OpenBSD root partition

2013-03-04T02:58:00-08:00

At ARP Networks, sometimes we have to resize an OpenBSD root partition to satsify a customer requirement, template requirement, or something else along those lines.

I’ve sometimes seen customers struggle to do this themselves, so the following is an example of how I resized a root partition from 5 GB to 9 GB, leaving 1 GB for swap. The underlying volume was already expanded from 5 GB to 10 GB, to give us room to do this.

So, let’s get to it.

Boot into the system’s rescue disk:
```
> boot bsd.rd
```
Then select (S)hell at the prompt.

Resize:

Grab the growfs(8) program from the root partition. This isn’t included with the rescue disk, for some reason.

# mount /dev/wd0a /mnt
# cp /mnt/sbin/growfs .
# umount /mnt

Re-initialize the MBR, using the entire disk.

# fdisk -i wd0
Do you wish to write new MBR and partition table? [n] y
Writing MBR at offset 0.
#

Change the size of the slices.

This system has only the root partition and swap. The idea is to remove the swap partition, then extend root, and finally put the swap partition back.

# disklabel -E wd0
# Inside MBR partition 3: type A6 start 63 size 20964762
Treating sectors 63-20964825 as the OpenBSD portion of the disk.
You can use the 'b' command to change this.

Initial label editor (enter '?' for help at any prompt)
> p
OpenBSD area: 63-20964825; size: 20964762; free: 9446220
#                size           offset  fstype [fsize bsize  cpg]
  a:         10490382               63  4.2BSD   2048 16384    1
  b:          1028160         10490445    swap
  c:         20971520                0  unused
> d
partition to delete: [] b
> p
OpenBSD area: 63-20964825; size: 20964762; free: 10474380
#                size           offset  fstype [fsize bsize  cpg]
  a:         10490382               63  4.2BSD   2048 16384    1
  c:         20971520                0  unused
> c
partition to change size: [] a
Partition a is currently 10490382 sectors in size, and can have a maximum
size of 20964762 sectors.
size: [10490382] 9G
Rounding to cylinder: 18876312
> a b
offset: [18876375]
size: [2088450]
FS type: [swap]
> p
OpenBSD area: 63-20964825; size: 20964762; free: 0
#                size           offset  fstype [fsize bsize  cpg]
  a:         18876312               63  4.2BSD   2048 16384    1
  b:          2088450         18876375    swap
  c:         20971520                0  unused
> w
> q
No label changes.
#

Grow the filesystem using growfs(8)

# growfs /dev/wd0a
We strongly recommend you to make a backup before growing the Filesystem

 Did you backup your data (Yes/No) ? Yes
new file systemsize is: 4719078 frags
Warning: 216792 sector(s) cannot be allocated.
growfs: 9111.1MB (18659520 sectors) block size 16384, fragment size 2048
        using 45 cylinder groups of 202.47MB, 12958 blks, 25984 inodes.
super-block backups (for fsck -b #) at:
 10781088, 11195744, 11610400, 12025056, 12439712, 12854368, 13269024, 13683680, 14098336, 14512992, 14927648, 15342304, 15756960, 16171616, 16586272, 17000928, 17415584, 17830240,
 18244896
#

Give it a rinse.

# fsck /dev/wd0a
** /dev/rwd0a
** Last Mounted on /
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
100205 files, 418854 used, 4172217 free (97 frags, 521515 blocks, 0.0% fragmentation)

MARK FILE SYSTEM CLEAN? [Fyn?] y


***** FILE SYSTEM WAS MODIFIED *****
#

And that’s pretty much it. Everything is done within OpenBSD’s rescue disk kernel bsd.rd, no need to have a separate CD-ROM / USB rescue disk utility.

Having a Javascript moment with Ruby, WTF is going on here...

2013-03-01T18:55:00-08:00

WTF is going on here?

(rdb:1) sum
244.4
(rdb:1) amount
244.4
(rdb:1) sum != amount
true
(rdb:1) sum.to_f
244.4
(rdb:1) amount.to_f
244.4
(rdb:1) sum.to_f != amount.to_f
true
(rdb:1) sum == amount
false
(rdb:1) sum.to_f == amount.to_f
false
(rdb:1)

First guess is some type conversion problem…

Still investigating.

Using the quota command within a chroot environment

2013-03-01T03:50:00-08:00

I was recently using Jailkit on Ubuntu to set up a chroot environment for users of a backup storage server. I wanted to give the users the ability to check their quotas with the regular quota command.

I go about telling Jailkit that the chroot directory will need the quota binary. This should be all that is needed, or so I thought.

Upon checking a quota as a chroot’d user, it barfs the following error,

$ quota
quota: Cannot open quotafile /home/aquota.user: Permission denied
quota: Quota file not found or has wrong format.
$

From what I can tell, this is a very misleading error message. The permissions of aquota.user are always 0600, readable / writable only by root. This can’t be changed, not even as root. Witness.

-rw------- 1 root root 7168 2013-02-27 22:32 /home/bkusers/home/aquota.user

Using a bit of strace and some luck, I find what the quota command is really looking for is /proc/sys/fs/quota.

It simply does a stat() system call on that directory within proc, so if you simply do,

$ sudo mkdir -p /home/bkusers/proc/sys/fs/quota

then the quota command works like a charm within the chroot environment.

$ quota -s
Disk quotas for user testuser (uid 1007):
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
       /dev/md3      16       0   3907M               4       0       0
$

My chroot directory in these examples was obviously /home/bkusers, modify accordingly for your needs.

The 'Yes' men

2013-02-17T02:47:00-08:00

Let me turn a corner and cover a non-technical topic here for once…

Several years ago, I had a job where I had to ask myself the following question, “Do I do what I think is right, or do I do what will make my new boss happy?”

Now, this was not a moral question, it was simply about how to do some measurements / benchmarking and systems stuff; pretty low key, no big deal.

I stuck to my guns and at some point shortly after, I didn’t have that job anymore. ;)

In my case, there really wasn’t any big life altering changes to myself or anyone else regardless of which decision I made. But in some cases, this can really go the wrong way…

Let me illustrate by quoting something from my reading last night; the book was Amy Goodman’s (Host of Democracy Now!) ”Breaking the Sound Barrier”, and within was a quote from Grammy Award-winning soul singer John Legend when he gave the commencement address at the University of Pennsylvania,

Too often, in business and in government, people are rewarded for having the answer that the person they report to wants them to have: “Yes, sir. We can provide mortgages to people who have no down payment and can’t afford the monthly payments.”… “Yes, ma’am. I can write a legal brief to justify torture.”
John Legend Commencement Address at the University of Pennsylvania 2009

Stick to your guns, fuck your boss.

Migrated from Mephisto to Octopress

2013-02-12T02:01:00-08:00

I’m proud to say I’ve dusted off this blog and migrated from Mephisto to Octopress. Now I can actually post new content (the old Mephisto system got so dated and went to dependency hell that I was hardly able to do anything besides keep the static cached files up).

I’m going to try to go through all the posts and cleanup / fix any loose ends. So far the import went pretty smooth and I’ve only burned a few hours (instead of a whole weekend).

I was unable to successfully import the comments, so those will be gone forever. Most were spammy anyway, and any comments posted at least within the last year or two use Disqus, and those are still around.

I’d like to get my usual theme working again, but alas, that is very low priority right now. Perhaps a new look is warranted anyway. However, at least, I can move away from the default theme. Gotta read more docs…

Let me know if anything is out of the ordinary.

Thanks!

Testing Octopress...

2013-02-12T00:38:00-08:00

Testing 1 2 3…

is this thing on?

Running OpenBSD 4.5 in KVM on Ubuntu Linux 9.04

2009-10-04T13:16:00-07:00

Here’s a quick post on how to get OpenBSD 4.5 working as a KVM guest on kvm-84, the version that ships with Jaunty.

Unlike OpenBSD 4.4, which worked out of the box, 4.5 made some changes in their kernel with respect to interrupts that may have exposed bugs in KVM/QEMU. The workaround is to disable mpbios within the OpenBSD kernel. This will affect SMP guests adversely, so it isn’t for everyone.

Your kernel is affected by this if you boot and it hangs at:

setting tty flags

Credit goes to Todd Fries of Free Daemon Consulting, LLC for figuring this out and passing down the wisdom onto me.

Note: I’m using a 64-bit host as well as the 64-bit version of OpenBSD as the guest.

Step 1

Boot the OpenBSD 4.5 install CD within your blank VM and proceed through the entire install normally. Then, reboot from the hard disk but stop at the boot prompt…

Step 2

At the boot prompt, perform the following:

boot> bsd -c
...
UKC> disable mpbios
 54 mpbios0 disabled
UKC> quit
...

System will now boot normally.

This has temporarily disabled mpbios and will not persist across reboots. You now have to make this change permanent in the OpenBSD kernel, as illustrated in the next step.

Step 3

# config -ef /bsd
OpenBSD 4.5 (GENERIC) #2052: Sat Feb 28 14:55:24 MST 2009
    deraadt@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC
Enter 'help' for information
ukc> disable mpbios
 54 mpbios0 disabled
ukc> quit
Saving modified kernel.
#

Now your system will boot up normally from this point onward.

New server

2009-10-04T11:01:00-07:00

Moved this blog to a new and faster server. Maybe now I’ll start posting again, since the admin section isn’t slowed to a crawl any more with spam comments. :(

If anyone notices anything missing, blow up, etc… Please leave a comment. Thanks!

Prevent Javascript onClick event from bubbling up

2009-04-05T23:53:00-07:00

I recently was working with some code that had an onClick event handler on the <TR> of each row of a table. In one of the columns, a <TD>, there were icons that also had onClick’s on them. Something like:

<table>
  <tr onClick='foo()'>
    <td onClick='bar()'>
      <!-- some icons -->
    </td>
    <td></td>
    <td></td>
    ...
  </tr>
</table>

Now if the icons in the first <TD> were clicked, I wanted bar() to fire, but not foo(). Normal behavior is to execute them both.

Turned out there is a simple solution for this with jQuery. Only tested in Firefox, but presumably this works across the board because jQuery is awesome that way.

Add a class to the <TD> called no-event-bubble-up:

  <td onClick='bar()' class='no-event-bubble-up'>

And add the following JS to your page:

<script type='text/javascript'>
  $(document).ready(function() {
    $(".no-event-bubble-up").each(function() {
      $(this).click(function(e) {
        e.stopPropagation();
      });
    });
  });
</script>

That’s all that is needed and it worked for me.

Let me know if you find this useful.

A Pledgie for Pledgie

2008-11-02T00:36:00-07:00

Pledgie is trying to go to the 2008 Philanthropy Midwest Conference.

But we need your help to get there!

We need to raise $500.00 over the next couple days to reserve a booth. We learned just yesterday that there was one last exhibitor booth available.

All the details are in our Pledgie 4 Plegie campaign.

Mark and I thank you for any support you can provide.

Get rxvt-unicode with 256 color support on Ubuntu

2008-10-13T05:15:00-07:00

rxvt-unicode (commonly called urxvt) already has 88 color support, and for most things, this is fine. But I recently just found the CSApprox plugin for vim, which lets you use Gvim themes in console vim. CSApprox actually does a pretty good job interpolating for 88 colors too, but it is best at 256.

So here’s my build log of compiling rxvt-unicode with the 256 color patch on Ubuntu Hardy, and debianizing (packaging) it.

# I keep all custom deb's here, use any directory you want
cd ~/debian-src

# Make a place for rxvt-unicode
mkdir rxvt-unicode
cd rxvt-unicode/

# Get the source
apt-get source rxvt-unicode
cd rxvt-unicode-8.4/

# Apply 256 color patch, it's included with the source
patch -p1 < doc/urxvt-8.2-256color.patch

# Make sure you have all depdencies to build it
sudo apt-get build-dep rxvt-unicode

# Build it
dpkg-buildpackage -us -uc -rfakeroot

This will actually build three separate packages:

rxvt-unicode
rxvt-unicode-lite
rxvt-unicode-ml

All I care about is rxvt-unicode, but you might want -lite or -ml. If someone more adept in Debian package building than I am can tell me how to just compile one of these versions, that’d be great. :)

Moving along:

# Install it!
cd ~/debian-src/rxvt-unicode
sudo dpkg -i rxvt-unicode_8.4-1_amd64.deb

# As always, your arch may be different.  Replace amd64 with whatever is
# appropriate for you.

Now we need to make terminfo aware that rxvt-unicode supports 256 colors. The default terminfo entry you’ll have for rxvt-unicode will say 88, and that won’t be right for your new version.

cd ~
infocmp -L rxvt-unicode > rxvt-unicode.terminfo

vi rxvt-unicode.terminfo
# Change the following from:
#
#    lines_of_memory#0, max_colors#88, max_pairs#256,
#
# to:
#
#    lines_of_memory#0, max_colors#256, max_pairs#32767

# Make .terminfo dir if you don't already have it
install -d .terminfo

# Rebuild terminfo for rxvt-unicode
tic -o .terminfo/ rxvt-unicode.terminfo

# Cleanup
rm rxvt-unicode.terminfo

That should do it.

To test it out, use this 256colors2.pl script (it’s the same one included in the xterm distribution). Download it, set it executable, open a fresh urxvt terminal, and run:

$ ./256colors2.pl

The output should look like:

Any program that supports 256 colors that had to fallback to 88 or even 8 colors can now be used in all its glory.

Does this make your consoling experience better? Let me know, post a comment.

TextMate-like file navigation in Vim

2008-10-11T13:05:00-07:00

I highly recommend the new fuzzyfinder_textmate Vim plugin.

In addition, this post which debuts the plugin, has a ton of other useful Vim tips and trick. If you’re a Vim user, I’ll bet you’ll learn something you didn’t know!

Hardy Xen domU doesn't show login prompt [fix]

2008-10-01T22:35:00-07:00

I have a Xen machine running Ubuntu Hardy (8.04.1). I created a guest that was also Hardy, with xen-tools. But after booting it up, the login prompt wouldn’t appear. It would get as far as:

...
 * Starting OpenBSD Secure Shell server sshd                             [ OK ] 
 * Running local boot scripts (/etc/rc.local)                            [ OK ]

And then just not show anything. The VM wasn’t hung, just the login prompt didin’t display.

After googling to no avail, I lucked out with button mashing some options in my <vm>.cfg file. Adding this to the end of my <vm>.cfg made it work:

extra = 'xencons=xvc console=xvc0'

Update: Immutable, with exceptions

2008-09-21T11:47:00-07:00

This is an update to my previous article: Making methods immutable in Ruby.

By popular demand, I’ve made my Immutable module raise an exception, by default, if you try to override an immutable method.

So, the following:

require 'rubygems'
require 'immutable'

module Foo
  include Immutable

  def foo
    :foo
  end

  immutable_method :foo
end

module Foo
  def foo
    :baz
  end
end

Will raise an error:

Cannot override the immutable method: foo (Immutable::CannotOverrideMethod)

Making methods immutable in Ruby (or, Death to Monkey Patching)

2008-09-17T12:52:00-07:00

Update (09-21-2008): I have a followup to this article here.

You’ve probably been told that in Ruby classes are always open and any code can come along and redefine your methods in any way it pleases. Indeed, this is true.

Kinda

What’s not true, despite every reference I’ve read saying the contrary, is that you can’t prevent this from happening.

Ruby provides a method_added() callback that is invoked every time a method is added or redefined within a class. It’s part of the Module class, and every Class is a Module. There are also two related callbacks called method_removed() and method_undefined().

This means you could detect when other code has redefined a method, and do something about it! How about redefining that method (again) to point back to your original code? Indeed, this works.

I’ve encapsulated the details of this in a new module I call Immutable. It provides one class method called immutable_method(). Provide it a list of methods you don’t want touched and it’ll make sure they can’t be redefined. Hence, immutable.

It’s on GitHub: http://github.com/up_the_irons/immutable/tree/master

Here’s an example:

require 'rubygems'
require 'immutable'

module Foo
  include Immutable

  def foo
    :foo
  end

  immutable_method :foo
end

# Now re-open Foo and redefine foo()
module Foo
  def foo
    :baz
  end
end

include Foo

foo # => :foo

foo() returns :foo, not :baz, even though we redefined foo() to return :baz.

If this makes you think of Java’s final method modifier, you’d be almost right in saying immutable_method is similar. But there’s one important difference: with immutable_method you can still redefine methods in subclasses. This makes sense, as one can argue that if you want to reimplement or extend a method, a child class is the only place where you should be doing that anyway. I tend to agree, call me a purist.

Why would this be useful to you? I don’t know. But I can tell you what motivated me.

My motivation for writing something that provides what some may call “evil” functionality (heh, and monkey patching isn’t evil?) of closed methods/classes came from my trials of trying to improve Rails loading time by reimplementing some ActiveSupport methods in C (Ruby extension).

I absolutely needed my C versions of certain methods to be present, and not clobbered by ActiveSupport.

So why didn’t I just load my extension after ActiveSupport? Because by then it is too late. Rails is already loaded, along with slow running methods that get called over 10,000 times during initialization.

Run the profiler and see for yourself:

./script/performance/profiler "require 'config/environment'"

This is a prelude to my next post, when I present the plugin I’ve built that takes away over 30% of overhead from Rails loading time. Stay tuned (or subscribe :)

But for now, if you want to try Immutable, clone it from GitHub, or if you have RubyGems 1.2.0 or higher, do this:

# Requires RubyGems 1.2.0+ (or so I'm told...)

gem sources -a http://gems.github.com
sudo gem install up_the_irons-immutable

Then you can run the example above. You’re all set.

See the README for further details.

Enjoy (or flame me for ruining your monkey patching).

Nicely formatted JSON on the command line

2008-09-03T01:36:00-07:00

You have a URL that outputs JSON, and you want to quickly see its contents from the shell, but nicely formatted, not the raw output wget spits; just do:

$ wget foo.com/bar.json -O - --quiet | ruby -rubygems -e \ 
  'require "json"; puts JSON.parse($stdin.gets).to_yaml'

I broke it up into two lines for easier reading, but you can type it all out in one line, or better yet, turn it into a shell script with a single URL argument.

GNU screen with vertical split support

2008-08-22T00:56:00-07:00

Just a quick post here. I recently recompiled screen with vertical split support on Ubuntu 8.04. Here’s how you do it:

# Install dependencies to build screen
sudo apt-get build-dep screen

# Create an area to hold the source
cd ~/debian-src
mkdir screen
cd screen

# Get the source
apt-get source screen

# Apply vertical split patch
cd screen-4.0.3/
wget http://vsp4sdl.yuggoth.org/wrp_vertical_split_0.3_4.0.2.diff.bz2
bunzip2 wrp_vertical_split_0.3_4.0.2.diff.bz2
patch -p1 < wrp_vertical_split_0.3_4.0.2.diff

# Build it!
dpkg-buildpackage -us -uc -rfakeroot
cd ..

# Install it!
sudo dpkg -i screen_4.0.3-0.4ubuntu2_amd64.deb

That’s it.

Automatically setting hostname as GNU screen window title

2008-08-19T21:25:00-07:00

I’m starting to put this in ~/.bashrc of all my servers now:

# Set screen window title
case "$TERM" in
screen)
  PROMPT_COMMAND='echo -ne "\033k$HOSTNAME\033\\"'
  ;;
esac

When I log into the machine, the screen window title (which displays on the hard status line, and in turn my urxvt window title bar) automatically changes to the hostname of the machine I’m logged into. I used to set this manually, but now I don’t have to, yay!

If your distro doesn’t set up your environment so ~/.bashrc is sourced upon login, you’ll want to put this into ~/.bash_login instead.

To get this code on a lot of servers at once, save the above snippet to a file called “prompt-command.sh” and execute:

for i in hostname1 hostname2 hostname3; do cat prompt-command.sh | ssh $i 'cat >> ~/.bashrc'; done

Of course, change “hostname1”, “hostname2”, etc… to the real hostnames of your machines.

Cool, huh?