The Solutious Blog

So I made over 52,000 mistakes today

2013-02-06T00:00:00+00:00

Earlier today I updated the net-ssh family of Ruby gems and I broke one of the rules of semantic versioning.

Specifically, rule #8:

8. Minor version Y (x.Y.z | x > 0) MUST be incremented if new,
backwards compatible functionality is introduced to the public API.

I broke Chef. I broke Vagrant. net-ssh is pretty far upstream so in just a couple hours there were over 52,000 installs of the offending gems, much to the chagrin of sysadmins and devops folks everywhere.

Note: If you have any of the following gems installed on your system, you should remove them: net-ssh-gateway-1.1.1, net-ssh-gateway-1.1.2, net-ssh-multi-1.1.1, net-ssh-multi-1.1.2, net-scp-1.0.5, and net-scp-1.0.6. See my previous post.

The err of my ways

I released three gems with the PATCH incremented instead of the MINOR version number. This makes a huge difference downstream because of the “twiddle-wakka”:

# Meanwhile, in chef.gemspec
s.add_dependency "net-ssh", "~> 2.2.2"
s.add_dependency "net-ssh-multi", "~> 1.1.0"

The ~> will fuzzily match any gems less than 1.2 but greater than or equal to 1.1.0. This feature strikes a balance between ">= 1.1.0" (which is too loose) and "= 1.1.0" (which is too strict). The problem is that net-ssh-multi-1.1.2 changed the net-ssh dependency to 2.6.5 which made Chef uninstallable due to the conflict between chef.gemspec and net-ssh-multi.gemspec (2.2.x vs 2.6.5). Feels bad man.

So if I ruined your day, send me your email, Twitter, Skype, or phone number and I will reply with a personal apology.

(Offer expires Feb 12th at 07:59 UTC).

On a more positive note

A big thank you to everyone who emailed, tweeted, and opened issues to help get this resolved quickly. Although regrettable, this is the only significant issue with net-ssh and friends in the 4 years (and 18M downloads) that I’ve been maintaining them. I feel pretty good about that.

Incidentally, I updated the THANKS.txt that’s part of every net-ssh release today too. I added the names of all the people who contributed code since I’ve been maintaining it. Here they are:

GOTOU Yuuzou
Guillaume Marçais
Daniel Berger
Chris Andrews
Lee Jensen
Hiroshi Nakamura
Andreas Wolff
mhuffnagle
ohrite
iltempo
nagachika
Nobuhiro IMAI
arturaz
dubspeed
Andy Brody
Marco Sandrini
Ryosuke Yamazaki
muffl0n
pcn
musybite
Mark Imbriaco
Joel Watson
Woon Jung
Edmund Haselwanter
robbebob
Daniel Pittman
Markus Roberts
Gavin Brock
Rich Lane
Lee Marlow
xbaldauf
Delano Mandelbaum
Miklós Fazekas
Andy Lo-A-Foe
Jason Weathered
Hans de Graaff
Travis Reeder
Akinori MUSHA
Alex Peuchert
Daniel Azuma
Will Bryant
Gerald Talton
ckoehler
Karl Varga
Denis Bernard
Steven Hazel
Alex Holems
Andrew Babkin
Bob Cotton
Yanko Ivanov
Angel N. Sciortino
arilerner@mac.com
David Dollar
Timo Gatsonides
Matthew Todd
Brian Candler
Francis Sullivan
James Rosen
Mike Timm
guns
devrandom
kachick
Pablo Merino
thedarkone
czarneckid
jbarnette
watsonian
Grant Hutchins
Michael Schubert
mtrudel
and of course, Jamis Buck.

I know I’m not the only one who appreciates your time and effort. Thank you for making net-ssh better!

All future Net-SSH gem releases will now be signed (as of 2.6.5)

2013-02-06T00:00:00+00:00

**Updated (2013-02-06@13:00PST): Doh. Some previously updated gems were broken. See below. **

In response to the recent vulnerabilities with rubygems.org, I spent the morning signing and re-releasing the Net-SSH family of ruby gems. The discussion on how to properly handle code signing is still ongoing so this could be just an interrim measure; however, the severity of the problem makes it necessary to have a solution in place now.

Current Signed Releases

As of today, all net-ssh releases will be signed and verifiable with the public certificate at the end of this post.

net-ssh 2.6.5+ (rubygems, github)
net-scp 1.1.0+ (rubygems, github)
net-sftp 2.2.0 (rubygems, github)
net-ssh-gateway 1.2.0 (rubygems, github)
net-ssh-multi 1.2.0 (rubygems, github)

Installation

You can still gem install net-ssh like you do already but if you want to verify the gem is authentic, you will now be able to run:

$ gem install net-ssh -P HighSecurity

To do this, you need to add the public certificate to local trust gem certs (otherwise you’ll see an error like "Couldn't verify data signature"):

$ curl -O https://raw.github.com/net-ssh/net-ssh/master/gem-public_cert.pem
$ gem cert --add gem-public_cert.pem

Broken versions

The following gems were broken:

net-ssh-gateway-1.1.1
net-ssh-gateway-1.1.2
net-ssh-multi-1.1.1
net-ssh-multi-1.1.2
net-scp-1.0.5
net-scp-1.0.6

They’ve been yanked from rubygems.org but if already have them on your system, you will need to remove them manually.

$ gem uninstall -v 1.1.1 net-ssh-multi
$ gem uninstall -v 1.1.2 net-ssh-multi
$ gem uninstall -v 1.1.1 net-ssh-gateway
$ gem uninstall -v 1.1.2 net-ssh-gateway
$ gem uninstall -v 1.0.5 net-scp
$ gem uninstall -v 1.0.6 net-scp

If you have any trouble let me know at net-ssh@solutious.com.

Public certificate

How RethinkDB Says Thanks

2012-12-26T00:00:00+00:00

I posted a couple weeks ago about my experience installing RethinkDB. Today I got this in the mail:

That’s a moleskin and a usb key (with a metal case). The handwritten note is fine touch too.

Thank you @al3xandru and RethinkDB.

Installing RethinkDB on Ubuntu

2012-11-12T00:00:00+00:00

Update: The installation docs for Ubuntu have been updated to make the requirements more clear.

RethinkDB has been getting some more attention in the past few days, starting with a Hacker News post last Friday. The project (and company) was founded in 2009, launched the 1.0 release last year, and although there’s still some work to do in terms of stability and portability, it looks promising.

In their words:

RethinkDB is built to store JSON documents, and scale to multiple machines with very little effort. It has a pleasant query language that supports really useful queries like table joins and group by, and is easy to setup and learn.

Installation Steps for Ubuntu

I only just got it running after much trial and error so I documented the experience to help others in the same boat. The team is working full steam on resolving issues as they arise so my intent here is one of observation rather than to criticize. Here’s what I’ve gathered so far.

Requirements/Limitations

Only runs on x86_64 Linux
Depends on kernel 2.6.37 or greater
No RPM (soon)
~~Possible issue with ext4 with journaling and/or encrypted partitions~~
(update from Slava below)
~~Some compilation issues on Debian/CentOS~~
(working on a Debian post now)

Update (Nov 12 @ 16:50 PST): Success with ext2 and ext3/ext4 with journaling enabled.

Update (Nov 13): Comment from Slava (below):

We’ll be providing binaries for most platforms and even old kernels, though it will take a bit of time to work out. The portability team here is working around the clock to make it happen.
Rethink doesn’t work on encrypted file systems because they don’t support direct io. It makes sense in production, but it’d be nice to have a backup mode for trying in dev. See issue #47.

So make sure you’re running the 64-bit of version Ubuntu with a non-encrypted ext2, ext3 or ext4 partition and that the kernel version is 2.6.37 or greater. Then:

$ sudo apt-get install software-properties-common  # installs add-apt-repository
$ sudo add-apt-repository ppa:rethinkdb/ppa
$ sudo apt-get update
$ sudo apt-get install rethinkdb

Things are changing almost hourly so be sure to check their community page and follow RethinkDB on Twitter for the latest info.

Checking whether journaling is enabled

Look for the has_journal option in the following output:

$ sudo debugfs -R features /dev/sda1
debugfs 1.42.5 (29-Jul-2012)
Filesystem features: filetype sparse_super

You may need to change /dev/sda1 for your disk device which you can get from the output of mount.

Checking the kernel version

The kernel version is displayed immediately to the right of the hostname:

$ uname -a
Linux bs-dev-01 3.5.0-17-generic #28-Ubuntu SMP Tue Oct 9 19:31:23 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

Some cool uses of Git-like hashes in Ruby (with Gibbler)

2011-03-11T00:00:00+00:00

Cryptographic hashes are pretty cool. They’re often used as checksums for large files because they’re fast, consistent, and well, secure. A lot of opensource software packages are distributed with the MD5 or SHA1 hash so that you can verify that all the bits are in the correct place (i.e. that the file you downloaded is identical to the one being served). If you’ve used Mercurial or Git, you’ve seen them used there too to track commits, objects, and trees.

Hashes can also be a useful tool in your code and I wrote Gibbler to make it easy to do that. Why not use Ruby’s hash method? Because the return values are inconsistent between runs.

t1 = Time.now
t2 = t1.clone
t1.hash                       #=> -2827223250544534006
t2.hash                       #=> -2827223250544534006 (the same!)
t1.object_id                  #=> 2170505820
t2.object_id                  #=> 2170481360

# Later on, with another instance of Ruby
t1 = Time.now
t1.hash                       #=> 2265941047042223117 (different!)
t1.object_id                  #=> 2168957700

But as it turns out, you can do some neat stuff when you can rely on the values between runs, using different versions and implementations of Ruby. I’m going to point a few, but first a quick introduction.

A quick intro to Gibbler

When you require gibbler, you get a gibbler method installed into most rudimentary Objects like String, Symbol, Hash, Array, etc.

require 'gibbler'

'tea'.gibbler                 #=> 6ef1ccef723f8f6c048399cfa5f46a781f559137
:tea.gibbler                  #=> 4f7721e1a1e0a02f87b196fd78f94358293793c1
{:count => 100}.gibbler       #=> 19322962506419bd16d9de2ab3d1e5ec0772c4e6
[4, 3, 2, '1'].gibbler        #=> b05b4fada2105f0f9547ae320423deba729abe53

Gibbler works similarly to Git: for complex objects, it dives depth-first and creates digests for each object and at each level creates a summary digest. The final digest is based on the summaries for each element.

[4, 3, 2, 1].gibbler          #=> d1cf67fb93ec51885e7c74e4b3a3d5ef3aad2bf9
[3, 2, 1].gibbler             #=> 18410df1574242b2730144ed483930072e49bd23
[3, [2, 1]].gibbler           #=> a05a76617a3b848060e6e8024e9c38a264dbd31b
[3, [2, [1]]].gibbler         #=> b32e17d4bf10eb7101153703511d08de4509e0ce

You can also include gibbler in your own objects with Gibbler::Complex which will create the hash based on the values of the instance variables:

class Email
  include Gibbler::Complex
  attr_accessor :to, :from, :subject, :content
  def initialize *args
    @to, @from, @subject, @content = *args
  end
end

msg1 = Email.new             
msg2 = Email.new 'd@example.com', 't@example.com', 'Hello', 'Long time no see!'

msg1.gibbler                  #=> 2667ed303e2e2cc307d49301acd7575ea3f90f2e
msg2.gibbler                  #=> 328dfe801c2563e31aa9a2b4831fa182f5e41dfd

By the way, if you prefer literal method names, you can require gibbler/aliases.

require 'gibbler/aliases'
'tea'.digest                  #=> 6ef1ccef723f8f6c048399cfa5f46a781f559137

A few examples of using hashes in your code

Know when a complex object has changed

When you store a record to your database, keep track of the latest hash. Later on, you can check that value to determine whether the contents of the object have changed without checking each field individually. You can also use the value of the hash to detect and prevent duplicate content. Here’s one example:

class Article
  include Gibbler::Complex
  attr_accessor :author, :title, :content, :checksum
  gibbler :author, :title, :content
  def initialize *args
    @author, @title, @content = *args
  end
  def changed?
    @checksum != gibbler
  end
end
article = Article.new 'jodie', 'Chicken Soup', '...'
article.checksum = article.gibbler
article.save

# Later on, in another process... 
article.content << "and it was delicious."
article.changed?              #=> true

Detect duplicate messages

You don’t need to store copies of an object to know if you’ve seen them before:

from = 't@example.com'
seen = []
['cust1@example.com', 'cust2@example.com', 'cust1@example.com'].each do |to|
  msg = Email.new to, from, 'A catchy subject', 'Some interesting content.'
  if seen.member?(msg.gibbler)
    # cust1 has already received that specific email
    next
  end
  seen << msg.gibbler
end

Find data without storing an index

I use this approach extensively for Stella (my web monitoring service). When a customer runs a checkup, it creates a testplan to represent the site and page being tested. I create a new instance of the object every time, but because the digest for a given testplan is always the same I know where the object is stored without looking it up the based on the URI. As well, I include the customer ID in the digest calculation so that a each customer has their own instance of the testplan. You can see an example of that here:

A testplan created from my account.
A testplan created by an anonymous customer.

Notice that the list of recent checkups is different for each. I don’t need to do anything special for this. It’s just a freebie that comes along with using these hashes.

Know which local objects to sync remotely

If you have data in one location that you need to synchronize remotely (database records, files, etc) you can use the hashes to determine which objects need to be sent over. This is exactly how git determines what it needs to send to or receive from a remote repo. Of course you could simply keep track of the record IDs (in the case of a database) but by using hashes you get duplicate detection for free.

Maintain an index pointer for an Array without storing the contents

This example can seem arcane but I’ve found it useful on more than one occasion. Let’s say you have a list of values and you want to always process them in sequence. And for whatever reason you don’t store the values locally but every time you see this array of values you want to continue processing at the appropriate element.

With hashes it’s simple: create an index using the gibbler hash of the array. It will always be the same as long as the values and the order of the values are the same (you could optionally create the hash after sorting the array).

indexes = {}
5.times do
  people = %w[dave john candace]  # inside the loop to simulate different arrays
  indexes[people.gibbler] ||= -1
  indexes[people.gibbler] += 1
  indexes[people.gibbler] = 0 if indexes[people.gibbler] >= people.size
  current_idx = indexes[people.gibbler] 
  puts people[ current_idx ]
end
# Output:
# dave
# john
# candace
# dave
# john

There are many more uses for hashes in your Ruby codes. I’m interested to hear some. Do you implement them in your projects?

Installing Gibbler

gem install gibbler

code at Github
gem on Rubyforge
documentation via RDocs
screencast by Alex Peuchert

Mini-F.A.Q.

Can digests be made unique per application?

Yep. Set Gibbler.secret to anything, preferably something long.

:kimmy.gibbler                #=> 52be7494a602d85ff5d8a8ab4ffe7f1b171587df

Gibbler.secret = '4cea880a75df6c8b1fa2'

:kimmy.gibbler                #=> 0f71d5813687cb07f8b6be5389e636962f49e213

What if attributes are added or removed to a class?

Use the gibbler class method to explicitly define the names and order of variables you want to use for the digest.

class Email
  include Gibbler::Complex
  gibbler :to, :subject, :content   # only these fields will be considered
end

msg = Email.new 'd@example.com', 't@example.com', 'Hello', 'Long time no see!'
msg.gibbler                   #=> 7f68056cf34cd42cbb3dee1f81535100ae783fe9

msg.from = 't2@example.com'
msg2.gibbler                  #=> 7f68056cf34cd42cbb3dee1f81535100ae783fe9

Can I use something other than SHA-1?

Yep, you can change the digest type globally or per call.

:a.gibbler                     #=> cd55a626c21b5580141442e789201e7e64276da9

Gibbler.digest_type = Digest::MD5
:a.gibbler                     #=> ef8de1a0d178ce85999a4b54840c21e0

:a.gibbler(Digest::SHA256)     #=> f5fa26a66724c32df25f872fd691dd18e03cc2347a...

You can also shorten and change the base of the digest:

:kimmy.gibbler                #=> 52be7494a602d85ff5d8a8ab4ffe7f1b171587df
:kimmy.gibbler.shorten        #=> 52be7494a602d85ff5d8
:kimmy.gibbler.shorten(10)    #=> 52be7494a6
:kimmy.gibbler.base(36)       #=> 9nydr6mpv6w4k8ngo3jtx0jz1n97h7j
:kimmy.gibbler.base(10)       #=> 472384540402900668368761869477227308873774630879
:kimmy.gibbler.to_i           #=> 472384540402900668368761869477227308873774630879
:kimmy.gibbler.base(2)        #=> 101001010111110011101001001010010100110000...

An update on yesterday's Stella outage (it was the Redis configuration)

2011-03-10T00:00:00+00:00

After I reported on the Stella outage yesterday, I did a bit more investigation and made a few changes to the operations of the site. To recap, yesterday at 9am PST the site became unresponsive and the background workers stopped running. I also couldn’t SSH in to the main backend machine and ultimately had to reboot it to get back in. After looking into it further, it looks like there were multiple factors at play.

Root Cause, revisited

Memory swapping and blocking. There was a conflict between the hourly backup process and the regular operation of the site. The hourly cronjob is simple: it copies the redis.rdb file, gzips it, encrypts it, and uploads it to S3. The conflict arose during the copy. Redis was configured to run a background save every 5 minutes to the same redis.db. What I didn’t realize is that redis blocks the bgsave while that file is being copied. That file had grown a lot with the additional traffic over the past few weeks. I didn’t notice this issue previously because the cp and bgsave commands ran pretty quickly. With so much more data, both take longer so it was only a matter of time that this would happen.

Operational Changes

I made the following changes:

Disabled background saves in Redis
Enabled append only persistence
Rewrote the hourly backup to run explicitly run a Redis background save
Added an hourly process to tidy unneeded data to keep the backups smaller
Added a nightly process to run bgrewriteaof to keep the append only file tidy

The SSH issue

There was still the issue of the hanging SSH connection. Before the outage and after I rebooted the machine, SSH was fine. Also, I was able to open a connection to the redis server the entire time so it wasn’t the network. I’m not 100% sure but I have a suspicion that it was a kernel problem. That suspicion is based entirely on several modprobe errors in the system console (missing modules). The machine was running an EBS instance type. I have had miscellaneous boot and connection problems with the EBS instance types so I decided to go back to a trusty old instance-store machine image. I built a new machine image and launched a new instance that replaced the previous machine.

These changes have been in production for the past few hours. I’ll report back if I need to make any further modifications.

Morning Stella outage

2011-03-09T00:00:00+00:00

Update: I posted a follow-up on this outage.

Stella was down for about an hour and a half this morning, including checkups and monitors. I received a notification at 9am (PST) and the site returned to normal at 10:40am. Here is a screenshot of status.blamestella.com form this morning:

The symptoms

The web servers were not responding and the master backend machine was not available via SSH (I could connect but it would hang after authenticating). The connection to Redis was not affected.

The cause

I couldn’t verify that there was a single root cause. I couldn’t SSH in to the machine until after it was rebooted but it looks like something happened while Redis was running a background save. The only access I had was through redis-cli:

redis> bgsave
(error) ERR Background save already in progress
redis> save
(error) ERR Background save already in progress
redis> info 
...
aof_enabled:0
changes_since_last_save:67162
bgsave_in_progress:1
used_memory_human:2.21G
mem_fragmentation_ratio:1.44
...

My thinking at the time was that the process was swapping to VM so I waited patiently (not so patiently actually) to see if it would finish. There’s 7.5GB of RAM and Redis was only using about 2.5GB so there shouldn’t have been a need to swap but it’s possible (I’ve noticed that the redis-server process has used as much as 1GB more than what it report via info). As you can see there was some data in memory that hadn’t been written to disk yet so that was my top priority (changes_since_last_save). After about 20 minutes it was clear that there was something else going on and my top priority switched to getting the site back up. The data is stored on an EBS volume so my guess is that there was degraded IO performance. To be clear, I don’t suspect it was an issue with Redis itself. However, that still doesn’t explain why I couldn’t login via SSH so it’s possible there was a network issue at the same time (which I’ve experienced before in other EC2 available zones).

At that point I made the decision to reboot the machine. I also started the process of provisioning a new backend machine using the most recent backup (which was created an hour earlier). I did these in parallel in the event that the existing machine did not come back up. It did and after it ran an fsck I was able to get into the machine. I brought redis up, tested it, and then started the workers (which do the monitoring). Then I brought up the site after a few minutes.

Note: in the report you can see that the site came up briefly at 10am (PST). This was before the backend machine was restarted. The web servers could read and write to Redis but the monitoring and checkups jobs were not running.

Next steps

Switch to append-only backups to prevent the need for epic writes.
Create and practice a new recovery process to provision a new machine right away which can run off of a snapshot of the master’s EBS volume.

How I handled the traffic from a single Smashing Magazine tweet

2011-02-11T00:00:00+00:00

Smashing Magazine tweeted about Stella yesterday. It generated a lot of traffic and quality feedback (around 130 tweets) so I wanted to write a bit about what I learned. I was away from the helm at the time so there were definitely a few lessons learned!

A tweet by Smashing Magazine on February 10th, 2011

The Numbers!

I was surprised by how much activity can be generated from a single tweet.

In the first 24 hours

2629 visitors.
4018 checkups.
54 signups (2% conversion).

The next day (so far, midday)

449 visitors.
889 checkups.
9 signups. (2% conversion)

The 2% conversion rate is consistent with the burst of traffic last month from Hacker News. You can see both spikes below in the chart from Reinvigorate. Stella was also on Designfridge.co.uk earlier this week.

What went well

It only took me about 15 minutes to get the site in order once I was able to get in front of my laptop. In that time, I added another front-end machine and added a couple more background workers.
Morton and Tucker are working really well in multiple ways. I tried to respond to every criticism and I spread the effort between Morton, Tucker, @blamestella, and my account so that I wouldn’t overwhelm any one feed with a stream of replies. I (or Morton, rather) successfully turned, “It takes a while to load. How’s that for #irony?” into:

Short, interesting tag lines are effective. It seems obvious now but “is your site fast?” is the result of about 6 months of iteration. It’s a good tagline because it’s very short but it’s immediately apparent whether Stella would be useful to you. By making your own strong tagline, you’re also making it easier for people to tell your story.
Edit: I also need to give a huge shout out to the guys at the Still Brandworks. They’re the heavy-duty muscle behind the UI/UX which is generating a lot of the positive attention!

What could have gone better

Well, the ideal scenario involves the site remaining fast and responsive! The site was grinding for about an hour before I could tend to it. There were also a number of precautionary rate limiters that were set too conservatively (eg. the number of checkups that can be run over a given period of time). I applied a couple short-terms fixes for both issues and have a longer-term plan in the works.

I got caught away from the helm. Someone should always be near the helm. Of course this is a challenge for a one-man shop and although I don’t expect this to be the case for long, I’m going to upgrade to a smartphone soon so I can make quick changes regardless of where I am. My Samsung x820 has served me well over the years, but every journey must end.
Most of the signups were for the free plan. I need to do a better job getting people to the paid plans.

In summary

I had a great time yesterday handling the new traffic and responding to feedback. Welcome to everyone who signed up and thanks to all the people who tweeted, sent feedback to Tucker, and ran checkups. And thank you @smashingmag!

Why public monitoring is important

2011-02-09T00:00:00+00:00

There was an issue on Amazon EC2 yesterday that affected the response times for a non-trivial number of their customers (myself included). I first noticed the issue when blamestella.com began to get very slow (around 11:45 PST). I checked Stella’s public report for EC2 performance and other sites were having issues as well:

Stella EC2 Report on February 8th, 2011

The chart represents the average network latency (light grey), application latency (grey), and download time (dark grey) for all monitored sites on EC2 (71 separate sites at the time of this writing). You can see that the service is typically very stable with regards to HTTP response times so the issue yesterday was clearly an anomaly, but there was no information on Amazon’s status page:

Amazon’s Status Page

Amazon Web Services Status page on February 8th, 2011

I love Amazon. For them, this was probably an isolated problem that affected a relatively small number of their customers. However, the customers that it did affect were left in the dark. The natural response would be to assume that it’s a problem with your application until you start hearing reports for other sites. This is where the value of public monitoring becomes apparent and it’s one of the reasons I built Stella.

Leading by example

Heroku is built on EC2 and their service was affected yesterday as well. They do a great job handling operational issues by being responsive and open with their customers. Yesterday was no exception:

Heroku Status page on February 8th, 2011

Although they didn’t have answers as to the cause, simply demonstrating that they’re on top of things is reassuring and a sign of a responsible, well-run company.

All services go down

Public monitoring isn’t about highlighting the failures of a site or service. It’s about accepting the fact that all services go down from time to time. Public reports help site operators and they keep customers informed.

Gateway support for Rye, via Rye::Hop

2011-01-29T00:00:00+00:00

Thanks to a fork from Vincent Batts, Rye now has gateway support!

rhop = Rye::Hop.new('firewall.lan')
rbox = Rye::Box.new('filibuster', :via => rhop)
rbox.uptime     # => 20:53  up 1 day,  1:52, 4 users

Or:

rbox = Rye::Box.new 'filibuster', :via => 'firewall.lan'

Gateway support allows you to access machines through a tunnelled connection (similar to Net::SSH::Gateway). This functionality is available in the 0.9.3 release.

Installation

gem install rye --version 0.9.3

About Rye

Rye is a Ruby library that provides a safe, simple way to access Unix shells. You create an instance of Rye::Box and the methods you call against it are mapped to shell commands over SSH.

rbox = Rye::Box.new('hostname')
rbox.pwd                               # => "/home/rye"
rbox.uname :a                          # => "Darwin rye-stage 9.7.0 ..."

With the block syntax, Rye looks a lot like a shell script.

rbox.batch do
  cd 'redis-2.0.0-rc3'
  configure
  make
  make 'install'
end

See the Github repo for more examples.

Introducing Bone and the Bonery

2010-12-20T00:00:00+00:00

Whoa, whoa, whoa, whoa, whoa. There’s still plenty of meat on that bone. Now you take this home, throw it in a pot, add some broth, a potato. Baby you got a stew goin’. – Carl Weathers

Configuration management is hard. A lot of stuff is hard, but you don’t mind because you expect it. Managing configuration on the other hand seems like such an innocent and simple problem. However, the solutions often turn to be horrible, debilitating nightmares. Part of the problem for me is that the tools are really complicated. That’s a bit unfair to say because at some point there is a reason for that complexity (large projects, large teams, etc). But when your team is just a few people strong and when you don’t have the resources to dedicate to specific tasks, it really helps to have simple, straight-forward tools.

So when it comes to configuration, I use something a little more inline with Carl Weathers’ philosophy. It’s a command-line tool and Ruby library called Bone. I’ve been using it internally for about a year to store configuration and what I like to call remote environment variables. It’s only version 0.3 so it’s very basic, almost to a fault. However, what it lacks in features in makes up for by endeavouring to never ruin your day.

Bone: Remote Environment Variables

It’s essentially the simplest possible key-value store. Here’s a taste:

delano@localhost$ bone set dbmaster=192.168.1.1
192.168.1.1

delano@prodfe$ bone dbmaster
192.168.1.1

The values you set on the command-line are available in your Ruby code like regular environment variables.

require 'bone'

Bone[:dbmaster]      #=> 192.168.1.1
Bone[:dbmaster] = '10.0.1.1'
Bone[:dbmaster]      #=> 10.0.1.1

You can also pass values by STDIN.

delano@localhost$ <path/2/redis_server.conf bone set redisconf

I use bone everywhere. Some examples:

I store all my configuration files in there so I can make one simple command to get a copy. (keys: redis-server-2.2, nginx-0.7, etc…)
In backup scripts, I make a call to bone to store the latest path. (bs-backup-latest)
I store the ip addresses of various machines so when I start new machines the scripts that on them will always grab the updated addresses. (bs-redis-master, bs-prod-fe)

Behind the scenes

It works by communicating with an HTTP server, called Boned using a simple REST API. Since all the data is stored remotely, the values you set can be made available to any machine that can connect to the Boned server. The API is based on two regular environment variables: a token, which is like your username, and a secret which is used to sign each request (the signing is based on the one Amazon uses for EC2, S3, etc). There is a third environment variable to tell bone which server to use.

# In your ~/.bashrc or equivalent...
export BONE_SOURCE=https://api.bonery.com/
export BONE_TOKEN=YOURTOKEN
export BONE_SECRET=sUp3R5eKru7

The Bonery

I’ve also launched an alpha version of The Bonery so you can try it out without running the server yourself.

GET YOUR API KEY

It’s alpha as a I mentioned, so don’t rely on it for production just yet. But get your key, throw in some variables, a file, some configuration, and baby you got a stew goin’!

The 70% Solution

2010-10-18T00:00:00+00:00

I used to joke with my team at YellowPages, “if you can’t make it great, make it just good enough”. It kept being a joke until I started my own company, when I realized that it was a reasonable approach to running a startup.

Starting a company is very different than working in a larger organization. When you have a specific role, you can focus all of your attention on a limited set of tasks [1]. But in a startup, there’s too much to do and too few people to do it. It’s the amount of work and also the variety of tasks. There’s a lot to do outside of your areas of expertise so it’s easy to feel overwhelmed and under qualified.

What didn’t work:

Working myself to exhaustion. This obviously was not sustainable and it took an unreasonably long time to recover. It also didn’t increase the quality of my output.
Focusing on just a couple things. I wrote some great code that I couldn’t build a service around. It was fun and I became a better programmer but again, it wasn’t sustainable.
Doing nothing. This turned out to be the worst solution of the three. It sounded like a great idea and I got some rest but it’s no way to earn a living.

I’ll be honest: I’m not Bo Jackson. I realized that if I was going to feel good about my work and remain motivated over the long haul I’d need to redefine what it meant to be done something.

Just good enough (for now)

Every endeavour has an ideal outcome and an actual outcome. My mistake was comparing the actual outcome with the ideal one in my head and feeling bad that I wasn’t living up to my own expectations. So I started evaluating my performance against 70% of the ideal – not in order to legitimize mediocre work but to be realistic.

When you’re working on any sufficiently ambitious project, you need to find time somewhere. You can pretend you can do everything all the time and be exhausted. Or you can be honest with yourself and feel good about your work. You can always follow up with a second effort later on.

So try the 70% solution. It’s surprising how much you can accomplish by consistently achieving modest goals.

–

[1] Depending on the culture of the organization that can change. I call it “responsibility creep”. It feels a lot like working at a startup without any of the benefits.

Don't waste time writing tests

2010-09-07T00:00:00+00:00

I don’t enjoy wasted effort. Not many people do. When it comes to testing the code I write, I’ve often felt like I was wasting my time. And sometimes I was. Not often, but enough that it got me thinking, there’s got to be a better way.

Some problems I have with tests and testing

Writing tests for volatile code doesn’t help anyone. Names are going to change. Entire namespaces will be added or removed. You want to feel free to do make those changes.
Test code looks nothing like the how the code is actually used. Most test files look like crap. I’m sorry, but it’s true. Your code gets intertwined with stuff that has nothing to do with your project. There’s also a lot of repetition.
It’s difficult to determine whether the time was well spent. You definitely know when you should have spent more time testing (when something breaks) but it’s difficult to know when you’ve spent too much time. Or just enough. Or in the worst case scenario, you wrote a lot of tests, but not necessarily on the right areas.
Maintaining a test suite is a pain in the junk. If you’re working on a large, well-established project then yeah, you’re going to need lots and lots of tests. That’s how you institutionalize quality and you probably have enough people to handle the workload. But if you have a small team or a rapidly changing project, that testing suite is going to bog you down.

My solution: treat tests as sample code

It’s the most simple solution I can think without going over: write code exactly as it’s used in the wild and put the test definitions in comments. That way all the code you see is code that could appear IRL. Even in the worst case scenario where I have no way to know whether the time was well spent, the work doubles as documentation. That’s a huge win.

To get this to work in Ruby, I wrote a library called Tryouts. (Thanks to Martin Aumont and Alexis Sellier for working on various implementation ideas with me.)

Here’s an example:

require 'rye'

# Setup
@local_sandbox = File.join(Rye.sysinfo.tmpdir, 'rye-tryouts')
@lbox = Rye::Box.new 'localhost'


## upload file
@lbox.file_upload 'README.rdoc', @local_sandbox
@lbox.file_exists? File.join(@local_sandbox, 'README.rdoc')
#=> true

## download file
@downloaded_file = File.join(Rye.sysinfo.tmpdir, 'localfile')
@lbox.file_download File.join(@local_sandbox, 'README.rdoc'), @downloaded_file 
@lbox.file_exists? @downloaded_file
#=> true

## download to StringIO
content = @lbox.file_download File.join(@local_sandbox, 'README.rdoc')
content.class
#=> StringIO


# Teardown
@lbox.rm @downloaded_file

An example of the Tryouts test syntax that doubles as sample code. More examples.

And yep, it runs:

Notes on brand positioning statements

2010-04-05T00:00:00+00:00

A comment from Alistair Croll today made me realize I didn't have a brand positioning statement for one of the projects I'm working on. While creating one, I put some notes together and thought I would post them in case they're helpful for someone. If you don't have a brand positioning statement or you want to understand them better, definitely take a look at the links below too!

Definition

A brand positioning statement describes the "mental space" a brand should occupy in the minds of a target audience. It serves as an internal document which guides most of a company's marketing communications strategies, programs and tactics. The brand positioning statement focuses on the elements and associations which meaningfully set a brand apart from the competition. It is typically constructed in the following format: "To (target market), Brand X is the brand of (frame of reference) that (point of difference) because (reasons).

via Venture Republic

Process

Describe your customers (so you know what's important to them)

Define yourself in terms of your competition (to help you explain your product)

Explain your greatest benefit (the one thing that's most important to your most important customers)

Put it together to create your Brand Positioning Statement

via Dan Wilson - Web Marketing Today

Templates

FOR ,
OFFERS
THAT PROVIDES .

via Dan Wilson - Web Marketing Today

OUR
IS THE ONLY
THAT .

via the Small Business Marketing Guide

FOR
IS A
WHICH PROVIDES
UNLIKE
WHICH PROVIDES .

via the Small Business Marketing Guide

FOR THAT ,
WE OFFER .
UNLIKE , PROVIDES THAT .

adapted from a comment by Thomas Ptacek

Automate SSH key authorization with Rye

2010-02-25T00:00:00+00:00

I got annoyed with manually authorizing my public keys to ~/.ssh/authorized_keys and ~/.ssh/authorized_keys2 so I added a feature to rye to automate the process.

$ rye authorize user@example.com

The expected output looks something like this:

$ rye authorize user@example.com
Authorizing user@example.com
Passwordless login failed for user
Password: ************
Added public keys for: 
/home/user/.ssh/id_dsa
/home/user/.ssh/id_rsa
Now try: ssh user@example.com

Installation

$ gem install -V rye

You can generate an SSH keypair with the following command iff you don’t already have one (by default they’re installed to ~/.ssh/id_rsa or ~/.ssh/id_dsa):

$ ssh-keygen -t rsa

The Secret of Object#to_s

2009-09-22T00:00:00+00:00

There’s something about Ruby that I’ve wanted to know for a long time: where does the hexadecimal value in #<Object:0x000001009e60f8> come from? Today I finally went looking for the answer.

When in doubt, look at the code

The documentation for Object#to_s tells us that the value is based on the object id: “Returns a string representing obj. The default to_s prints the object‘s class and an encoding of the object id.”

Looking at the source code for to_s, we can see that it’s using sprintf to create the hexadecimal value. From object.c:

/* Ruby 1.9 */
rb_sprintf("#<%s:%p>", cname, (void*)obj);

/* Ruby 1.8 */
snprintf(RSTRING(str)->ptr, len+1, "#<%s:0x%lx>", cname, obj);

You can find this code in the Ruby source code in the file ruby-VERSION/object.c

Now, Ruby’s sprintf doesn’t support %p or %lx but it does support %x. However, there’s obviously more to the story because the values still don’t match up:

obj = Object.new
"#<%s:0x%x>" % [obj.class, obj.object_id]        # => #<Object:0x80813ff4>
obj.to_s                                         # => #<Object:0x101027fe8>

So what do we need to do to the object id to get the real hexadecimal value?

A tiny calculation

If we take a look at the hexadecimal values in plain-old decimal, the answer becomes obvious. Can you see it?

"0x80813ff4".hex                                 # => 2155954164
"0x101027fe8".hex                                # => 4311908328

All we have to do is double it!

And the answer is…

"#<%s:0x%x>" % [obj.class, obj.object_id.abs*2]  # => #<Object:0x101027fe8>

Note: I’ve created a tiny project called Hexoid to handle the minor formatting difference between Ruby 1.8 and 1.9.

Releasing with Rudy

2009-09-14T00:00:00+00:00

I have about 10 active opensource projects. I’m lucky because most of them aren’t that popular so there’s not a lot of overhead. Beyond other people using your work there are some significant advantages in maintaining projects as opensource (I’ll talk more about that in a future post). That said, there is additional effort involved so I’ve automated some tasks with Rudy to reduce the repetitive tasks. One such task is the release process.

Minor warning: The following is specific to Ruby development on Linux/Unix with git because that where I do most of my work. If you’ve created configurations for other languages (including projects on Windows) let me know and I’ll post them or include them in the arcade.

My Rudy Configuration

One of the core features of Rudy is the routines configuration. A routine is like a really concise Rake task that can be associated to a particular machine role. A little known fact is that Rudy also supports defining global routines which are available to all machines.

Rudy looks for global routines in two places:

~/.rudy/
/etc/ (linux only)

I put the following configuration into a file called ~/.rudy/release-routines.rb.

commands do
  allow :rm
end

routines do
  
  publish do
    before :package_test, :publish_docs
    local do
      project = File.basename pwd.first
      puts "PUBLISH #{project}", $/
    end
    after :publish_github, :publish_gem
  end
  
  package_test do
    local do
      puts 'Creating Test package...'
      rake 'package', 'clean'
    end
  end
  
  publish_docs do
    before :publish_rubyforge_docs, :publish_github_docs
  end
  
  publish_rubyforge_docs do
    local do
      puts 'Updating Rubyforge docs...'
      rake 'publish:rdoc', 'clean'
    end
  end
  
  publish_github_docs do
    local do
      rake 'rdoc'
      if file_exists?('doc')
        puts 'Updating Github Pages...'
        git 'checkout', 'gh-pages'
        rm :r, :f, 'files', 'classes' 
        unsafely { mv 'doc/*', '.' }
        rm :r, :f, 'doc'
        git 'add', '.'
        git 'commit', :a, :m, 'Updated docs'
        
        git 'checkout', 'master'
        git 'push'
      else
        puts 'No docs directory'
      end
    end
  end
  
  publish_github do
    local do
      puts 'Pushing to GitHub...'
      git 'tag', :f, 'latest'
      git 'push', '--all'
      git 'push', '--tags'
    end
  end
  
  publish_gem do
    local do
      puts 'Publishing Rubyforge gem...'
      rake 'publish:gem', 'clean'
    end
  end
  
end

The Rudy configuration I use to create releases of my open source projects.

My Release Process

With the configuration above in place, I can run the release routines I created from any directory on my machine. But before I run the automated process, there are a few tasks that still require manual attention.

Update the change log
Update file manifest and dependencies in the gemspec file (if necessary)
Increment the release number
Commit all local changes
Create a release tag

Then I run the following simple command from the project directory:

$ rudy publish

There’s huge value in being able to define a routine in one location yet be able to run it from each individual project directory. It establishes a standard process for all of your projects which reduces overhead (including the mental workload) and increases consistency.

For example, you’ll note the publish_github routine creates a tag called “latest”. That allows me to refer to that tag in long-running or rarely updated documentation when I talk about the most recent release. It’s little things like that that make a big difference over the long run. Both in terms of my time and the visible consistency of a project.

Future Additions

Astute readers will note a couple glaring improvements than can and should be made to this process. The first is creating the release tag automatically based on the gemspec (or possibly library version). The second is removing the reliance on Rake. There are some tasks that Rake is specifically suited for, but in this case it’s possible to cleanly migrate the calls for building the documentation and Rubygems into Rudy directly.

And that’s it. Do you automate your release process?

A need for virtualized infrastructure tools

2009-09-10T00:00:00+00:00

Virtualized infrastructure is packed with surprises. There is an interesting post on the Boxed Ice blog titled, Why we moved away from “the cloud” to a “real” server, that highlights two important ones:

Virtualized infrastructure (and Infrastructure as a Service or IaaS specifically) is not a direct replacement for physical hardware.
Moving to “the cloud” can increase development overhead.

Not a direct replacement, but a new opportunity (or three)

So if IaaS is not a direct replacement, how are companies using it today? There are currently three primary business cases for using IaaS (unless you’re a startup or otherwise have the luxury of starting new projects from scratch):

Ad-hoc Environments: Consider a testing environment that arrives just-in-time and only for the time that you need it. Instead of owning and maintaining physical machines for testing, you or your team can create machines on-the-fly, run your tests, and then shut the machines down. The advantage being you’re in control and you’re also paying only for the time the tests are running. There are also technical advantages in installing and configuring your application from scratch because it can help you find bugs that you wouldn’t find on permanent test machines.

Another example is rapid-prototyping. If your development or marketing team has an idea, you can spend a few days or a week building a prototype, then launch it on EC2 for internal and external feedback without having to plan and budget for the ongoing costs.

Batch Processing: Small to medium sized companies now have access to a massive amount of processing power. Before the IaaS model – that is, paying for computing power by the hour – this kind of power was available only large organizations which had the resources to purchase and maintain their own data centres.

Pilot projects: This one is important for development teams and IT managers in small and medium sized business. You now have the ability to provision your own hardware. While the costs can remain on par (in some cases), you will have more control and be more agile. More importantly you’re developing a knowledge of virtualized infrastructure within your organization. Just like the transition from on-site electricity generators to pay-by-usage utilities, the computing industry is transforming into a new model. You don’t need to move your entire infrastructure to experience this value. All it takes is 1 or 2 machines.

These examples are just the tip of the iceberg in terms of what’s possible, but they have one important thing in common: these are opportunities that you can employ today with minimal impact to your existing processes.

Development overhead, a need for tools

Possible? Absolutely. But these opportunities require (sometimes) significant development time to implement. That’s a sign that the ecosystem needs more tools.

Opensource tools: Most (if not all) programming languages have opensource libraries that implement the IaaS vendor APIs. But there are very few libraries that provide functionality beyond these core APIs. Rudy is one example that helps organize complex environments and processes, but there are other approaches. Like PoolParty and libcloud. This is a good start, but we need more.

Commercial products: Products like RightScale and Cloudkick which provide web-based interfaces to your virtual infrastructures. And there are companies like SOASTA and Devver which provide usage-based services on top of IaaS vendors. These tools are great and I’ve had success in training clients how to use them. But again, we need more.

The road is paved with experiments

We need to expand our knowledge of what works and what doesn’t work. So go out and experiment with EC2 or Rackspace or RightScale or Cloudkick. If you’re a development manager, green light a pilot project. If you’re a developer, take a look at a project like Rudy or PoolParty and tear it apart.

How do we get to a place that’s brimming with mind-blowingly powerful IaaS tools? Practice!

Restore, Work, Destroy: An EC2 lifecycle

2009-08-28T00:00:00+00:00

Most of the buzz around virtualized IT infrastructure focuses on dynamic scaling – that is, increasing the number of machines when load is high and decreasing when load is low. There is huge value in being able to do this, but there are other valuable opportunities which get less attention. The opportunity that I am most excited about is temporary infrastructures.

Now, temporary infrastructures have received some attention but the focus is generally limited to variations of batch processing. I’d like to talk about another possibility: launching applications only for the time that you need them.

A Just-in-time Issue Tracker

Launching an application, using it, and shutting it down is interesting but it’s hardly mind blowing. I’m going to kick it up a notch by demonstrating how to routinely persist an application and its data between machine instances using Amazon EC2, EBS volumes, and Rudy.

Imagine being able to run an issue tracker for your team, only during business hours. Off hours the machine is shut down (along with the applications running on it). Instead of paying $75 / month for the smallest EC2 instance, you’re now paying around $18. And there’s a bonus: you’ll be creating full daily backups as part of your regular process.

I’m using JIRA for this example because it is a popular bug and issue tracker that’s familiar to many people.

What you need

An Amazon Web Services (AWS) account and credentials
Some experience running tools on the command-line
Rudy installed on your machine
A JIRA License (optional)
5 minutes

Note: I am running this demonstration from OS X, but Rudy runs on Windows and Linux as well.

Initial Installation

After installing Rudy, download the configuration file for this demonstration. Before running the first command it’s important that you understand what it does:

Launch an instance of Linux in EC2
Create a user to run the JIRA application
Open port 8080 (for your IP address only)
Create a 10GB EBS volume
Download, install, and start JIRA

The meat of the configuration looks like this:

install do
  before :startup               
  adduser :jira                # Create a user on the linux
  authorize :jira              # machine to run the app.
  network do                   # Open access to port 8080
    authorize 8080             # for your local machine.
  end
  disks do                     # Create an EBS volume where
    create "/jira"             # JIRA will be installed.
  end
  remote :root do
    disable_safe_mode          # Allow file globs and tildas.
    
    raise "JIRA is already installed" if file_exists? '/jira/app'

    jira_archive = "atlassian-jira-standard-3.13.5-standalone.tar.gz"
    uri = "http://www.atlassian.com/software/jira/downloads/binary"
    wget "#{uri}/#{jira_archive}" unless file_exists? jira_archive

    cp jira_archive, '/jira/jira.tar.gz' 
    cd '/jira'
    mkdir :p, '/jira/indexes', '/jira/attachments', '/jira/backups'
    tar :x, :f, 'jira.tar.gz'
    mv 'atlassian-jira-*', 'app'
    chown :R, 'jira', '/jira'
    ls :l
  end
  after :start_jira
end

When you’re ready, run the following command:

$ rudy --config path/2/jira.rb install

Once this command finishes, you’ll be able to grab the the public DNS for your new EC2 instance by running:

$ rudy --config path/2/jira.rb machines

JIRA Setup (optional)

This step requires a JIRA license. You can generate a trial license or skip to the next section.

Open JIRA in your browser by copying and pasting the public DNS info into the address bar. Make sure to specify port 8080! It should looking something like:

http://ec2-75-101-174-156.compute-1.amazonaws.com:8080/

Continue through the setup wizard and you should end up with a page that looks like this:

You can use JIRA as much as you like. Create a project, file a bug, etc… In the next section we will destroy this instance and later, through the magic of television, restore it back to the state you left it.

Destroy!

This is the exciting part where we destroy the instance and the disk! This next command will:

Stop the JIRA application
Create a snapshot of the disk (EBS volume)
Destroy the disk (forever)
Shutdown the instance (also forever)

The relevant configuration looks like this (note how the destroy routine refers to dependencies for the first two steps):

destroy do
  before :stop_jira, :archive
  disks do 
    destroy "/jira"
  end
  after :shutdown
end

archive do
  disks do
    archive "/jira"
  end
end

Take a deep breath and run:

$ rudy --config ruby/unix/jira.rb destroy

The instance is now gone. You can try to reload JIRA in your browser but sadly and perhaps slightly frightening, it’s gone.

Restore

And finally the moment we all been waiting for: restoring JIRA to its previous glory. This command will:

Create a new EC2 instance
Restore the disk from the most recent backup
Start the JIRA application

Here is the relevant configuration:

restore do
  before :startup
  adduser :jira
  authorize :jira
  network do                   # Open access to port 8080
    authorize 8080             # for your local machine 
  end                          
  disks do                     # Create a volume from the
    restore "/jira"            # most recent snapshot
  end
  after :start_jira
end

Run the following command:

$ rudy --config ruby/unix/jira.rb restore

Note that even if you skipped setting up JIRA, it’s in the same state as you left it without having to reinstall in. If you did setup JIRA, you’ll notice that it’s in the same state as you left it too. However, notice how the public DNS is different. That’s because it’s an entirely new machine with an entirely new disk.

Don’t forget to destroy the instance again when you’re done!

Final Thoughts

This was a very simple demonstration. What is most interesting to me is that backups are implicitly included in this lifecycle. Not only does it cut the cost of running a machine by over two thirds (assuming business hours), it provides complete daily backups as part of the regular process.

And of course, if you were to use this configuration, there are other things you’d probably want to do (install MySQL, modify the JIRA configuration, etc…) but it already gives you a look into what is possible with temporary infrastructures.

What do you think: would this kind of workflow be useful for you?

Running SSH on Windows

2009-08-19T00:00:00+00:00

SSH (Secure Shell) is a network protocol that allows data to be exchanged securely between one or more devices. It’s used to automate command execution and transfer files. SSH is most often associated with Linux, Unix, and other “non-Windows” operating systems, but it can run on Windows as well. This document takes you through the steps for installing an SSH server on a Windows machine.

Overview

There are several SSH servers available for Windows, both free and commercial products. I’ve found copSSH to be the easiest and also the most up to date.

Download the installation package
Run installer
Authorize users
Copy public key
Update Security Group (Amazon EC2 only)
Test connection
Bundle AMI (Amazon EC2 only)

Note: This post describes the process for enabling passwordless logins via SSH. You will not be able to login with your regular password.

Download the installation package

The following installation package must be downloaded to the Windows machine which will run the SSH server.

http://sourceforge.net/projects/sereds/files/copSSH/2.1.1/Copssh_2.1.1_Installer.zip/download

Run Installer

When you double click the installer, Windows will display a warning because the software isn’t signed by a known publisher. This is typical for free and opensource software on Windows. It’s safe to click Run.

Most of the screens explain themselves, but there is one the requires special attention. You need to specify a user with administrator rights to the machine. This example uses the Administrator account, but you can also specify a different one. It will create a new account for you if you specify a user name that does not already exist.

Note: Be sure to click Ok when prompted to start copSSH as a service.

Authorize users

After completing the installation, you must authorize SSH access for at least one user. You can do this by going to Start > All Programs > Copssh > 01. Activate a user.

Copy Public Key

In order to enable passwordless logins, you must copy your public keys to the Windows machine. You can find your default public keys in ~/.ssh but you can specify others as well. Here’s a basic command for creating an authorized_keys file.

$ cat ~/.ssh/id_rsa.pub ~/.ssh/id_dsa.pub >> ./authorized_keys

This file will contain a list of your public keys, one key per line. You must upload this file (or copy and paste the contents) to C:\Program Files\ICW\home\Administrator\.ssh\authorized_keys.

Update Security Group

For Amazon EC2 customers

You need to open access to port 22 for the security group associated to your Windows instance. You will need to replace the source IP address used in this example (192.168.0.1/32) with either your external IP address (very safe) or 0.0.0.0/32 (less safe because this means it’s open for the entire internet).

You can use one of the following methods.

Via Rudy

$ rudy-ec2 groups --authorize -p 22 -a 192.168.0.1/32 grp-name

Via Amazon API Tools

$ ec2-authorize -p 22 -s 192.168.0.1/32 grp-name

Via Amazon AWS Console

console.aws.amazon.com

Test connection

If everything went well, you will be able to login via SSH:

$ ssh Administrator@yourwindowsmachine
Last login: Wed Aug 19 07:01:19 2009 from 38.108.74.21

Administrator@yourwindowsmachine
$ 

Bundle AMI

For Amazon EC2 customers

If you’re running Windows within EC2, you need to create a machine image based on the running instance. Otherwise you’ll need to follow these steps again when you start a new instance!

There are two steps to bundling a Windows image. The first creates an image from the running instance and stores it to Amazon S3 (“Simple Storage Service”). Before executing the following command, you’ll want to consider the following:

Have you changed the Administrator password? If not, make sure you’ve written down the password you used to log in the first time. Amazon doesn’t keep it on record so if you don’t know the password, you won’t be able to log in!
Make sure everything on the machine is exactly you would like to see it when you start a new instance. For example, if you leave a bunch of files on the desktop that you don’t need, you will have to see them every time you launch a new instance of this image.
Empty the recycle bin.
Copy anything off of the D: drive that you want to keep. That is a temporary disk that is destroyed when the instance is shutdown and not stored in the machine image.

Then run the following command to initiate the bundling process.

$ ec2-bundle-instance i-12345678 -b BUCKET-IN-US -p IMAGE-NAME -o accessaccessaccess -w secretsecretsecret -K path/2/pk-***.pem -C ~path/2/cert-***.pem --url https://us-east-1.ec2.amazonaws.com --region us-east-1

Bundling can take anywhere from 20 to 60 minutes. You can check on the progress by running ec2-describe-bundle-tasks. Once it’s complete, run the following to register the machine image to your account.

$ ec2-register --region us-east-1 BUCKET-IN-US/IMAGE-NAME.manifest.xml

That’s it. Don’t forget to shutdown the Windows instance!

Other Resources

If you have any questions or problems, feel free to contact me via the discussion group.

Net::SSH 2.0.13 Released

2009-08-17T00:00:00+00:00

I pushed the 2.0.13 maintenance release of Net::SSH today.

Notable Fixes

arcfour128, arcfour256, and arcfour512 support (GH-4)
Support for specifying a list of hosts in .ssh/config (GH-6)
Fix for hanging in ServerVersion#negotiate! when using SOCKS5 proxy (GH-9)

Known Issues

I could use help with the following bugs:

Doesn’t work with Ruby 1.9 in Windows (GH-5)
PasswordAuthentication in ssh_config excludes all other auth methods (GH-8)
Net:SSH problem with open_channel (GH-1)

Installation

Via Rubygem, one of:

sudo gem install net-ssh
sudo gem install net-ssh-net-ssh --source http://gems.github.com/

Via .tar.gz or .zip

More Info

Thanks

I’d like to send a big thanks to Gerald Talton, ckoehler, Denis Bernard, and Karl Varga for their help.

This is my first official release as the new maintainer for Net::SSH. Please let know if you find any issues or problems.

Enjoy!

Rudy 0.9 Released

2009-08-10T00:00:00+00:00

After 8 months of development, testing, writing, and rewriting, Rudy has reached version 0.9. This is an important release because it establishes a baseline for the upcoming 1.0. The configuration syntax, command-line interface, and Ruby APIs are now stable and will not change. The road to 1.0 will be paved with testing, bug fixes, and documentation.

Note: Rudy 0.9 is not compatible with previous releases. See Upgrading.

What is Rudy?

Rudy is a development and deployment tool for EC2. It helps you build and maintain infrastructures by organizing them into groups of zones, environments, and roles. By making it quick and easy to build infrastructures, Rudy makes it feasible to run environments only for the time that you need them. Get started now.

Who is it useful for?

Rudy is useful to anyone starting a project with Amazon EC2. It’s also useful for launching test environments on-the-fly. My beta customers have been using Rudy to build and maintain their stage and production environments. I even use Rudy to script my release process (I’ll publish my configuration in a future post).

There are many possibilities. You can find other examples in Rudy’s Arcade and in my EBS test.

Configuration

Rudy is configured via several simple domain specific languages. There’s a machines configuration which describes the “physical” properties of your infrastructure and a routines configuration for describing repeatable processes (similar to short scripts).

The following is an example configuration that creates a machine called m-us-east-1b-stage-app-01. This machine has a 100GB EBS volume mounted at /rudy/disk1 and one user named rudy.

   
 machines do
   env :stage do                   # Define an environment
     ami 'ami-e348af8a'            # Specify a machine image
   
     role :app do                  # Define a role
       disks do                    # Define EBS volumes
         path "/rudy/disk1" do  
           size 100
           device "/dev/sdr"
         end                       
       end
     end

   end 
 end
 
 routines do
 
   startup do                      # $ rudy startup
     adduser :rudy       
     authorize :rudy               # Enable passwordless login
                          
     disks do                   
       create "/rudy/disk1"        # Create and mount a volume 
     end                       
                         
     remote :rudy do               # Run remote commands via SSH
       mkdir :p, "great"           # $ mkdir -p great
       mysql_init :start 
       your_script 'arg1', 'arg2'  # Call your own scripts
     end                          
   end
 
 end

See the default Rudyfile for a complete configuration sample

Installation

$ sudo gem install rudy

See Getting Started for details.

More Information

Fork at GitHub
Read the Documentation
Submit issues to the Issue Tracker
Start a discussion on the Discussion Group
Find some Inspiration

Thanks

In addition to everyone that has used and is using Rudy, I’d like to thank (in no specific order):

Kalin Harvey for the encouragement and answering my random questions.
The Rilli team for the initial usecase and invaluable feedback: Adam Bognar, Andrew Simpson, Caleb Buxton, Colin Brumelle.
Sam Aaron for teaching me about Ruby and communicative programming.
Everyone at Utrecht.rb for the interest and encouragement.
Steve Abatangle for the bug fixes.
Mathias Monnerville for allowing me to register Rudy at Google Code.
Jamis Buck for Net::SSH
Glenn Rempe for amazon-ec2 (and the quick releases!)
And of course, Keshia Knight Pulliam.

What’s Next

There’s a lot of exciting stuff in store for Rudy and Solutious. Look forward to more announcements in the coming months.

Command-line Twitter with Bash

2009-07-13T00:00:00+00:00

$ tweet "The sun is out and I am going to make a sandwich. #montreal #food" <span class="graphicSubtext"><a href="http://twitter.com/solutious/status/2620100947">Some minutes ago</a> from web</span>

I found a gist the other day by defunkt with a bash function for posting to Twitter from the command-line. It’s a cute hack but it was insecure so I updated it to work securely over HTTPS.

Step 1: Update ~/.bashrc

# $ tweet "your message"
#
# See: http://solutious.com/blog/2009/07/13/bash-twitter/
#
# Contributors
# * @defunkt for the initial version
# * @anildigital and @grundprinzip for curl-fu
# * @solutious for the SSL sexytime

# See: http://curl.netmirror.org/docs/caextract.html
CERTS_URI=http://curl.haxx.se/ca/cacert.pem

# SHA1 digest for file: Thu Mar 26 21:23:06 2009 UTC
# NOTE: This needs to be updated when cacert.pem is updated
CERTS_DIGEST=331b7e928bd758146ef4a17fee59a63e6ad6b10a

# Path to local copy of CA extract 
CERTS_FILE=~/.cacerts.pem

function tweet {
  verifycerts || return 1
  if [ ! "$*" ]; then
    echo 'Nothing to tweet!'
    return 1
  fi
  curl --cacert $CERTS_FILE -n -d status="$*" https://twitter.com/statuses/update.xml > /dev/null 2>&1
  echo "tweet'd '$*'"
} 

# Download extract of CA certs from mozilla.org 
function updatecerts {
  curl $CERTS_URI > $CERTS_FILE 2> /dev/null
  verifycerts
  echo "Saved to $CERTS_FILE"
}

# Make sure the CA certs match the sha1 digest
function verifycerts {
  if [ ! -f $CERTS_FILE ]; then
    echo "$CERTS_FILE does not exist."
    echo "Try running: updatecerts"
    return 1
  fi

  openssl sha1 $CERTS_FILE | grep $CERTS_DIGEST > /dev/null
  if [ $? != 0 ]; then
    echo "Digest mismatch for $CERTS_FILE (maybe it was updated?)"
    return 1
  fi
  return 0
}

Step 2: Update ~/.netrc

Put your twitter credentials into ~/.netrc. This file is read by curl.

# put this in ~/.netrc
machine twitter.com
login USERNAME
password PASSWORD

Then update the file permissions so only you can read it:

$ chmod 600 ~/.netrc

Step 3: Update SSL certs

$ source ~/.bashrc
$ updatecerts

This downloads a copy of the Mozilla SSL certificate authority bundle to ~/.cacerts.pem. curl uses this file to verify the SSL certificate for http://twitter.com/

Step 4: Tweet!

$ tweet "Anything at all"

Errors

Digest mismatch for $CERTS_FILE (maybe it was updated?)

This probably means that the Mozilla SSL certificate authority bundle has been updated. First check that the date in the bundle is later than “Thu Mar 26 2009”. If it is, then you need to do the following:

Download the latest copy of the bundle by running updatecerts (ignore the error)
Update CERTS_DIGEST in ~/.bashrc with the output from: openssl sha1 $CERTS_FILE

If the date has not changed, the ~/.cacerts.pem file has been modified. It may have been tampered with or corrupted. Delete it an run updatecerts again.

Contributors

If I missed anyone, please let me know!

A new repository for Net::SSH

2009-06-19T00:00:00+00:00

I use Net::SSH quite extensively in Rudy and Rye so I was concerned about the future of the project when Jamis Buck announced his departure from the project back in February. It’s a great library and without it I would not be able to do this:

script :root do
  tar :z, :x, :f, 'bonnie64.tar.gz'
  mv 'bonnie64', '/tmp/bonnie64'
  cd '/tmp/bonnie64'
  make 'SysV'
end

A configuration snippet from my Amazon EBS test demonstrating Rudy’s shell script style DSL.

A few weeks ago I ran into a connection problem with Solaris. The issue was related to the encryption modes supported by Net::SSH and was first reported back in January. Luckily a patch was recently posted by Denis Bernard which I reviewed and applied to my local copy. Problem solved.

Well, my initial problem was solved but not the greater problem of Net::SSH maintenance. I really appreciate the time Jamis has put in to Net::SSH and I want to make sure the library continues to be maintained.

A New Repository

I figured other people could benefit from this patch and there will likely be more patches to come so I forked the Net:SSH repository on GitHub with the canonical net-ssh user. I’ve applied the patch mentioned above, updated the test suite, and incremented the version to 2.0.12. I also added a gemspec file so the gem can be distributed via GitHub:

$ sudo gem install net-ssh-net-ssh --source=http://gems.github.com/

An Interim Maintainer

I have a hunch there are other developers out there who are willing and able to make contributions to the project. My intent is simply to help organize that effort. I plan to apply pull requests and patches but I do not plan to continue feature development myself at this time. For those interested in contributing, I’m happily accepting patches, pull requests, and even suggestions for minor changes. You can email me at net-ssh@solutious.com. If this is successful and/or helpful to the community, I’ll fork over Net::SCP and the rest of the crew as well.

A final note: I am not a security professional so if anyone has experience in this area and is interested in reviewing changes before they’re pulled into the repo please contact me.

How to put invite codes on business cards

2009-05-24T00:00:00+00:00

When I started working on Stella last year, my initial plan was to release it as a for-charge testing service. I figured out a way to put unique invite codes on business cards and about halfway through the project I got them printed and started handing them out to people I met at meetups and other events. It turned out to be a pretty handy way to punctuate a good conversation. The only problem was that about a month before launch I decided to switch gears and release the project as an open source tool. That was a good lesson learned! Don’t give out invite codes until the product is up and running [1]. In any case, I thought I’d share the process I used to create them for anyone else that’s interested in doing something similar.

The Process

Generate the invite codes
Batch create images
Print the cards with Moo

Generate the Invite Codes

I wrote a short Ruby script to generate the invite codes in the format in-XXXXXX. I added the in- to give them a little context and I chose six characters since it’s a good balance between uniqueness (31^6 = 887,503,681) and readability. I also removed the potentially ambiguous characters – i, l, o, 0, and 1 – which also doubles as a handy quick-check for bogus codes. Here’s the script:

#!/usr/bin/ruby

# Description: Invite code generator
# Usage: ruby genvites [code length] [count]
# 
# e.g.
#
# $ ruby genvites 6 3
# in-5qgczr
# in-enh4v4
# in-7jjab1

LENGTH = (ARGV.shift || 6).to_i 
COUNT  = (ARGV.shift || 50).to_i

# An Array with ambiguous characters removed: i, l, o, 0, 1
CHARS = [('a'..'z').to_a, ('0'..'9').to_a].flatten - %w[i l o 0 1]

def strand(len, str='')
   str << CHARS[rand(CHARS.size-1)]
   str.size == len ? str : strand(len, str)
end

COUNT.times do 
  puts "in-#{strand(LENGTH)}"
end

I created an invite codes file with the following commands:

$ echo "InviteCode" > invites-2009-05-27.csv
$ ruby geninvites 6 50 >> invites-2009-05-27.csv

Batch Creates Images

I used Photoshop so that’s the process I describe here but you can use any method that you prefer. You simply need to make 50 images, each with a unique invite code. Other options include Gimp, online image editors like Pixlr and Aviary or even a Ruby/Perl/Python script.

Step 1: Create a CSV file containing the invite codes. The first line of the file should contain the column names. If you’re using the script and commands above then you will have a file with one column named “InviteCode” and 50 codes (Moo has a limit of 50 unique images per batch of business cards). However you choose to generate the file, it should look like this:

InviteCode
in-92gr41
in-6imab7
...

Step 2: Create a new image and add a layer with some text in it. The styling of the text will be used for the invite codes. The dimensions of the image depend on the type of card you want to print. For a business card, I used 1040 by 700 at 300 dpi (see this discussion on Flickr). Check Moo Help for information on image sizes for all product types. Of course, you’ll also need to style the image however you like (with a background image or colour). Mine looked like this:

Step 3: Add a variable for that layer using Image > Variables > Define. The name of the variable needs to match the name of the column (i.e. “InviteCode”).

Step 4: Import the CSV file from Image > Variables > Data Sets or click “Next” in the Define screen. Click the “Import…” button and make sure the two checkboxes are checked (“Use first column” and “Replace existing datasets”).

Step 5: Generate the unique PSD files with File > Export > Data Sets as Files. The default settings will name the files in the format “original-name_invitecode.psd”.

Step 6: Convert the PSD files to JPG with File > Scripts > Image Processor. Select the input folder containing the PSD files, JPG encoding, and the quality.

Take it to the printers

It’s now time to start the printing process with Moo [2]. Their interface is pretty self-explanatory so I’ll only mention a few things here.

If you’re going with business cards, they have two batch sizes: 50 and 200. Obviously it’s cheaper per card to get 200, but they have a limit of 50 unique images (i.e. invite codes) so you’ll end up with duplicates (the other card types, like the mini cards allow up to 100 unique images).

They have several ways to upload photos, including via Flickr. I chose to go the manual route and upload all 50 photos to Moo. My guess is it’s probably easier to use a Flickr upload tool, but I haven’t tried it. After you’ve uploaded the photos, you’ll be asked to “Add Text”. You can enter your business name and contact info here using one of their preset templates or you can upload another image to use instead. If you’re uploading an image, you’ll need to use the full landscape template as shown here:

The only remaining choice is the type of paper: “Green” or “Classic”. A few observations: the 100% recycled paper is chlorine free and has a flat, matte finish. The classic is “elemental chlorine free” and is a bit shiny and fancier looking. I don’t understand the environmental implications of either choice so if I don’t consider the environment, I’d choose the classic. If the recycled paper does offer an environmental benefit I’d go with that. It’s not that the green is unattractive it’s just that the classic is more attractive.

Conclusion

And that’s how I put unique invite codes on business cards. It’s a bit of work, but it’s cheap and it gives people a reason to keep your card. Just make sure you launch the product!

Update: Improved ruby code example with feedback from Wes and Filip.

Notes

[1] I kept track of the codes that I gave out so they'll still work when we do release a product. It's kind of annoying to keep a business card around for an indefinite period of time so if you have one (or had one) just send me an email whenever the time comes and we'll figure it out.

[2] I recommend Moo because I don't know of any other printer that can produce business cards with unique images for $21.99 (USD). If there are others, let me know!

A Fine Brand: from ascii to awesome in 8 sweet months

2009-05-15T00:00:00+00:00

The new version of Solutious.com went live this week as part of a larger effort to develop a brand for the company and the products I’m working on. I’ve been working with designers Adam Bognar and Andrew Simpson and I’m really excited about how it turned out. I thought it would be interesting to take a look at how the site and brand developed over the past 8 months.

Prehistory

When I started the company in 2006, I put up what I thought would be a temporary homepage. I had an ongoing contract with a great company so I didn’t have an immediate need to update the site. Looking back, I probably had too many eggs in that one basket. It worked out okay, but the prudent choice would have been to spend a little time on business development regardless of the circumstances. In any case, the only information on the site for 2 years was the name of the company and a phone number.

October 2008, a rudimentary start

In 2008 I decided to shift the focus of the company from providing services to bootstrapping a product (more about that in a future post). I absolutely needed a website at this point and after a few months of intense research and prototyping I came up with this:

I was still heavy into product development so I had no content. There were no slogans and there was only a factual description of the company. I needed to put something on the homepage so I threw in a motivational Samuel Johnson[1] quote: “All industry must be excited by hope”. I hadn’t put much thought into it at the time, I simply thought it was interesting given the sombre tone of the economy, but as it turned out the spirit of that quote became the first kernel of the Solutious brand.

However, to say this design was an improvement over the stark yet earnest ASCII is a tough sell. It was more like a cry for help. I clearly required the services of a professional.

January 2009, the quick fix

That professional came in the form of Andrew Simpson. I hadn’t worked with Andrew before but we’d been friends for a while and I was impressed by his previous work. He was (and still is) incredibly busy as one of the founders of Rilli so I was fortunate to get some of his time.

I had a simple, two-pronged request: a quick fix and a plan for the future. I still had very little content for him to work with (and for some reason I decided to use a silly quote from Alistair Cockburn), so I was quite pleased with what he came back with.

I released the first public version of Stella around the same time and shortly after this design went up I began working on Rudy. You’ll notice the title on this homepage is “The Performance Testing Company”. That’s because my original focus was performance testing tools (I’ll talk more about the business side of things in that upcoming post). With the introduction of Rudy, a development and deployment tool, I removed the word “testing” and adjusted my approach to include the notion of human performance. Software shouldn’t just perform well at run-time, it also needs to perform well at development time.

March 2009, the prototype

The project really started to take shape in March. I had a much better understanding of both the products and the direction of the company so when Andrew brought Adam Bognar on board there was a lot more content to work with. Adam has a tonne of branding experience so I was excited to have him involved.

We had a couple conversations over Skype and I sent them a motley collection of copy (“Developing web applications in today’s world is no easy task”), old-timey advertising slogans (“Good Natured Tools”, “World Famous Components”) and examples of early advertising. I even sleuthed out a turn of the (19th) century theme song. Shortly after they created and built the prototype.

I was pretty stoked at this point. We had the content, we had logos, and for the first time in this process we had some focus. I was envisioning a future for Solutious based on this design. But they weren’t done.

May 2009, A Fine Brand

About a month or so after we went live with the prototype, Adam sent me this:

I was blown away. This design says everything I want people to know about Solutious. On top of that, there’s a wealth of visual components that allows for all manner of flexible and dynamic content. This blog for example fits in perfectly and I’ll be able to bring in the product documentation too. Andrew suggested using sIFR to render the text components. I didn’t fully understand the value until I started looking at the templates myself. If I want to change a slogan, create a new title, etc… I can make those changes on the fly. The intermediary step of creating and uploading an image is gone. Once the dust settles I may convert some static components to images, but at this stage that flexibility to make changes is extremely valuable.

On Branding

Branding and brand development have a bad rap outside of design and business circles. Brands tend to be viewed as a mere interpretation, often a false one, of what a company or product actually is or does. That’s fair because there are a lot of dubious and/or misleading brands (Fernandez, I’m looking at to you), but it’s not the only interpretation. It’s also possible and even advantageous to align a brand with reality. This is particularly valuable for small companies and micro-ISVs because it’s a way to align one’s personal and professional goals.

Conclusion

I can’t give enough credit to Adam and Andrew for creating this honest and cohesive brand from my haphazard reference material[2]. I wasn’t sure what to expect when we started out. I certainly had no idea that the end result would be something that motivated me to bring the company to the next level. I hope they got as much value out of the process as much as I did!

It was a fun, non-linear, and surprising project. If I’ve ever been excited about and hopeful for the future, it’s now.

Notes

[1] I ran a tiny, underground t-shirt business in 2006-2007. By “tiny” I’m referring to the fact that I had only one design and by “underground” I mean to say that the only sales channel was me wearing it and random strangers stopping me to ask about it. The shirt was a drawing of Louis Riel (Canada’s preeminent historical rebel) but when I was in the United States I’d say it was Edgar Allen Poe. I had several other designs in the pipeline, including one of Samuel Johnson (who wrote the first english dictionary by the way). It was going to be a drawing of him smiling while reading the newspaper with text below that read, “Samuel L Johnson”. If you’re into t-shirts and you’re in Montréal, I highly recommend Blank.

[2] I did make one contribution to the design – a sketch of the Stella logo. Fortunately the end result looks much less like two whales kissing.

What GitHub's Punch Cards Say

2009-05-06T00:00:00+00:00

I’m fascinated with GitHub’s punch cards. They’re visual representations of project commits by day and hour and they’re available for every public project on GitHub. Here’s the punch card for mislav-will_paginate, one of the most popular projects on GitHub:

will_paginate

The punch card says Mislav works hard! He committed at all hours of the day, every day of the week (the larger the dot, the greater number of commits). It’s important to note, when looking at the punch cards, how many active developers have committed to the project. You could look through the list of commits but that’s tedious and verbose. The quickest way to gauge how many active developers there are on a project is to look at the impact graph (note that there are several committers for will_paginate, but the vast majority are from two accounts, both of which are Mislav).

Punch Cards Over Time

I thought it would be interesting to see what a punch card looked like over time. I started saving them for a few of my own projects so I could animate them.

Rye (0.3.1 to 0.6.4)

Rye is a small project that I started on April 4th for executing SSH commands from Ruby. I wrote the core of it over a couple of weekends and then iterated over the following few weeks. You can see that in the animation below.

Rudy (0.4.0 to 0.7.3)

I also saved them for a much larger project, Rudy. I started Rudy in February while doing some deployment work for Rilli. The first frame is from version 0.4 and the second frame is version 0.5. They’re so different that it would be easy to assume I made a mistake generating the animation. That difference represents a month of hard work where I wrote and re-wrote the project several times.

In Conclusion

What I love about the punch cards is that they’re both informative and motivational. They give me a better understanding of how I work and how my effort relates to that of other open source developers. When I’m focused on a goal like getting the next version out the door, I occasionally forget what I went through to get there. The punch cards help me remember.

Louise Hendy Day, a Celebration of Realtime Marketing

2009-05-05T00:00:00+00:00

If you’re like me, you woke up today thinking May 5th was Cinco de Mayo. What people like us didn’t know is that it’s also Louise Hendy Day. This is probably the first and last time you’ll hear about Louise Hendy Day, but the implications of it are pretty interesting.

A Fictitious Holiday

Louise Hendy Day is a fictitious holiday created by a single person, earlier today, in an attempt to get into the trending topics on Twitter. And it worked. At least for a few hours (it’s no longer a trending topic as of 16:00 EST but the flurry of messages continues).

Why is this important? It’s important because we’ve heard a lot about the power of the web, “social media” in particular, to reach out to build and maintain networks of clients and customers. Rarely however, do we see this power in action, in real time. One person generated a massive buzz literally out of nothing. The product is a faux-celebration for a holiday that doesn’t otherwise exist.

A New Way to Do Business

Now, there are two ways you can view Louise Hendy Day: a useless, passing fad or a completely new way to do business. You can communicate with current and prospective customers in real-time. It’s like having a conversation with thousands of people at the same time.

Twitter won’t do this alone – there will be other service providers – but already today, right now we have a tool more powerful than any PR agency ever had. That’s pretty exciting.

Goodnight Sweet Prince (Dreamhost)

2009-05-03T00:00:00+00:00

I had a difficult time putting to words the excitement I felt when GitHub announced CNAME support. And now, just a few short months later I’ve finally freed myself from Dreamhost’s shackles. I’m now using GitHub for all my hosting needs (including solutious.com) and I must say, it feels pretty good.

Shared hosting has served a useful purpose over the years, but it’s days are numbered. Heroku: free. Google App Engine: free. AppJet: free. You can’t argue with that. GitHub is not free but CNAME support is included with all paid accounts which I already have. I’m now riding the workflow of the future!

Workflow of the Future

Me -> TextMate -> Markdown -> Terminal -> Jekyll -> Git -> You

It's an outrake!

2009-04-30T00:00:00+00:00

This is a story about Ruby mixins, Rake, and hell. After releasing Rudy 0.6 last week, I started immediately on 0.7. This next version has a rebuilt DSL for the routines configuration, including a new syntax for running shell commands. The new syntax makes it possible to specify shell commands like methods. You can probably see where this is heading. First, a comparison of the old and new syntax:

Old, 0.6 syntax

routines do
  startup do
    after :root => "mkdir -p /path/2/create"
  end
end

New, 0.7 syntax

routines do
  startup do
    after :root do
      mkdir :p, "/path/2/create"
    end
  end
end

There are several advantages to using the new syntax which I’ll cover in a future post. Right now, I’ll cut to the hellish chase.

Both DSLs produce a configuration hash. Nothing else should happen when they’re parsed. And nothing else did happen until I tried running the tests with rake. The old syntax was fine, but I was getting some strange errors with the new syntax. Stuff like

Error in test/10_config/30_machines_test.rb: can't convert Fixnum into String
/usr/local/lib/ruby/1.9.1/fileutils.rb:1386:in `path'

FileUtils? That’s weird, I don’t use FileUtils. Or do I?

rake-0.8.4/lib/rake.rb includes FileUtils into the RakeFileUtils module and later includes RakeFileUtils into the global namespace. My criminy!

Offensive ruby code

module RakeFileUtils
  include FileUtils       # Nothing wrong here
end
# ... later on
include RakeFileUtils     # Noooooooo

That means when I’m running rake the following methods are accessible from everywhere: cd, mkdir, touch, rm, …! The mkdir in the DSL was being executed as it was parsed. That’s insane. Thank the Great Scott I wasn’t testing rm.

Rudy, A Replacement for EC2 API Tools.

2009-04-22T00:00:00+00:00

NOTE: This post is historically accurate but out of date. See: Rudy 0.9 Released.

Rudy is a development and deployment tool for EC2. It comes with an executable called rudy-ec2 that can also be used as a replacement for Amazon’s EC2 API tools. The EC2 API tools are a great reference implementation but they’re unfriendly and become unwieldily when you have more than a few instances. rudy-ec2 is an alternative to these tools that can be used on it’s own (you don’t need to start using Rudy), allowing you to incorporate Rudy into your development process gradually.

Installation

rudy-ec2 is included with Rudy so if you’ve already installed Rudy, you can skip to the next section.

Download Rudy from github.com or install it via RubyGems:

$ sudo gem install rudy
$ sudo gem install solutious-rudy --source http://gems.github.com/

NOTE: If you are not installing via RubyGems and running Ruby 1.8.x, you need to make sure the Ruby dependencies (see the README) are installed and in your LOAD_PATH. Ryan Tomayko wrote a gist about it.

Configuration, 3 ways to do it

rudy-ec2 needs your Amazon Web Services credentials in order to execute commands on EC2. You can configure these in 3 ways: rudy configuration, environment variables or command-line options.

Rudy Configuration (method 1)

rudy-ec2 can use the same configuration as rudy. If you’ve already created a Rudy configuration file, you don’t need to do anything else. Otherwise, run the following commands:

$ rudy generate-config
  [edit ~/.rudy/config with your Amazon Web Services credentials] 
$ rudy init

NOTE: The above command is rudy and not rudy-ec2.

Environment Variables (method 2)

Bash

Add the following to your ~/.bashrc file:

AWS_ACCESS_KEY='your_aws_key'
AWS_SECRET_KEY='your_aws_secret'
AWS_ACCOUNT_NUMBER=123456789012
export AWS_ACCESS_KEY AWS_SECRET_KEY AWS_ACCOUNT_NUMBER

EC2_CERT=path/2/cert.pem
EC2_PRIVATE_KEY=path/2/pk.pem
export EC2_CERT EC2_PRIVATE_KEY

Then either start working in a new terminal window or run source ~/.bashrc to refresh your current one.

Windows/DOS

Set your environment variables from your System Properties menu.

Command-line Options (method 3)

$ rudy-ec2 -A your_aws_key -S your_aws_secret COMMAND

NOTE: If you use the command-line options, your credentials will appear in your shell history. You can run history -c to clear it.

rudy-ec2 Commands

A rudy-ec2 command follows the same convention as rudy commands:

$ rudy-ec2 -h
USAGE: rudy-ec2 [global options] COMMAND [command options]

The default command displays the current instances

$ rudy-ec2
i-8794fcee  ec2-11-22-33-44.compute-1.amazonaws.com  (group-awesome)
i-9e94fcf7  ec2-55-66-77-88.compute-1.amazonaws.com  (group-awesome)
i-9394fcfa  terminated  (group-awesome)

When no arguments are given, all commands display information about the object you specify:

$ rudy-ec2 images
Images owned by amazon
aki-46e7002f i386   (aki-linux.2.6.21.7-2.fc8xen-xfs/vmlinuz.manifest.xml)
aki-9800e5f1 x86_64 (ec2-public-images/vmlinuz-2.6.18-xenU-ec2-v1.0.x86_64.aki.manifest.xml)
aki-9b00e5f2 i386   (ec2-public-images/vmlinuz-2.6.18-xenU-ec2-v1.0.i386.aki.manifest.xml)
...

$ rudy-ec2 groups
group-awesome (authorized accounts: 123456789012:default)
11.22.33.44/32 -> tcp(22), tcp(80), tcp(443)
55.66.77.88/32 -> tcp(22), tcp(80), tcp(443)

With arguments, you can create, modify or destroy all EC2 objects:

$ rudy-ec2 instances -C -m ami-235fba4a -s m1.small -k keypair-name
i-9394fcfa  pending  (default)

$ rudy-ec2 groups -A -p 8080,8081 group-awesome
Authorize access to group-awesome from 11.22.33.44/32
on tcp ports: 8080, 8081
default (authorized accounts: 123456789012:default)
   11.22.33.44/32 -> tcp(22), tcp(80), tcp(443), tcp(88), tcp(99)
   11.22.33.44/32 -> tcp(8080), tcp(8081)

Safety

I’m always a little nervous using the Amazon AMI tools because I feel like I’m one command away from a nightmare. rudy-ec2 solves this by prompting for user input before executing any potentially destructive actions.

$ rudy-ec2 groups -D group-awesome
Destroying group: group-awesome
Are you sure? To continue, resolve (7 * 5): 

And you can avoid the annoyance by providing the -Y global

$ rudy-ec2 -Y groups -D group-awesome    # Careful!

Conclusion

That’s rudy-ec2 in a nutshell. For more information, check out the RDocs.

Rudy, not your grandparent's deployment tool

2009-04-21T00:00:00+00:00

NOTE: This post is historically accurate but out of date. See: Rudy 0.9 Released.

Rudy is a command-line development and deployment tool. It’s been in private beta for about two months and today marks the day of the first public release [1]. The project is still in alpha status and many features have yet to be developed but it’s already useful for exploring the features of Amazon EC2. In this post, I’ll introduce Rudy’s core features and demonstrate how these features can help build and manage environments in EC2.

A quick look

It’s useful to know what working with Rudy looks like before getting into the details:

$ rudy startup            # Launch the default machine group, stage-app
$ rudy -e dev startup     # Launch the dev-app machine group
$ rudy machines           # List all instances running in stage-app
$ rudy -u root ssh        # SSH to root@m-us-east-1b-stage-app-01
$ rudy ssh uptime         # Execute the uptime command on app stage-app machines
$ rudy shutdown           # Terminate stage-app instances

Machine Groups

Rudy helps you build and manage machines in EC2 by organizing them into groups of environments and roles. These are called machine groups. You can run multiple machines with the same role. These are called positions. Rudy also supports running machine groups across availability zones. When you put all this together, you have a unique name for every machine. The default machine is:

         zone     env  role
          v        v    v   
    m-us-east-1b-stage-app-01
    ^                       ^
 "machine"                position

stage is the default environment and app is the default role, but these can be changed too (see Defaults below). These machine groups are configured using a Ruby DSL (domain-specific language) that looks like this:

machines do
  # Define two identical environments
  environment :dev, :stage do
    # Default properties for all roles
    ami "ami-235fba4a"
    positions 1
    
    role :app do
      # Create two identical instances in the :app role. 
      positions 2
      # Elastic IP addresses to be associated on startup
      addresses "11.22.33.44", "55.66.77.88"
    end
    
    role :analyzer do
      # The :analyzer machines can have their own AMI
      ami "ami-********"
      # Define EBS volumes for this machine
      disks do
        path "/data/disk1" do
          size 4
          device "/dev/sdr"
        end
      end
    end
    
  end    
end

The configuration above describes the properties for 4 machine groups: dev-app, dev-analyzer, stage-app, and stage-analyzer. Since we’ve defined two app machines per environment, that makes 6 machines:

m-us-east-1b-dev-app-01
m-us-east-1b-dev-app-02
m-us-east-1b-dev-analyzer-01
m-us-east-1b-stage-app-01
m-us-east-1b-stage-app-02
m-us-east-1b-stage-analyzer-01

Routines

The machines configuration describes the whats or the “physical” characteristics of the environments. The routines configuration describes the repeatable processes.

routines do
  # Define routines for the stage environment
  environment :stage do

    role :analyzer do
      # Tell Rudy what to do when you run "rudy startup"
      startup do
        # Execute "uname" on the local machine
        before_local Rudy.sysinfo.user => [:git, :commit, 'a', 'm', "Rudy Startup"]
        before_local Rudy.sysinfo.user => [:git, :push]
        disks do
          # Create an EBS volume, attach it, format it, and mount it
          create "/rudy/disk1"
        end
        after :username => ['/a/release/script']
        after :root => ['/etc/init.d/nginx', 'restart']
      end
      shutdown do
        before :root => '/another/custom/script'
        before :root => [:mysqladmin, :u, 'root', 'shutdown']
        disks do
          # Umount the volume, detach it, delete it
          destroy "/rudy/disk1"
        end
        # Rudy.sysinfo.user returns the process' current user (ENV['user'])
        after_local Rudy.sysinfo.user => "date"
      end    
    end
    
  end
end

This routines configuration describes the processes for the stage-analyzer group.

Running a Routine

$ rudy -r analyzer startup
Starting stage-app

---  BEFORE SCRIPTS (local)  ------------------------------
Connecting to localhost
...
Starting m-us-east-1b-stage-app-01
Associating 11.22.33.44 to i-11111111
Starting m-us-east-1b-stage-app-02
Associating 55.66.77.88 to i-22222222
....

---  DISK ROUTINES  ---------------------------------------
...

---  AFTER SCRIPTS  ---------------------------------------
...

The following machines are now available:
m-us-east-1b-stage-app-01  ec2-11-22-33-44.compute-1.amazonaws.com
m-us-east-1b-stage-app-02  ec2-55-66-77-88.compute-1.amazonaws.com

Defaults

You may be wondering where the us-east-1b value comes from. This is an EC2 availability zone and it’s one of several default values that Rudy assumes in order to function straight “out of the box”. The default values can be changed too:

defaults do
  region :"us-east-1" 
  zone :"us-east-1b"
  environment :stage
  role :app
  position "01"
  user ENV['USER'].to_sym
end

Conclusion / More to come

As I mentioned, the current release of Rudy (0.6) is in alpha status so it’s not ready for production. There’s a lot of work to do, including a lot of documentation to write, but hopefully you can already see the value of incorporating Rudy into your development process. For now, the README contains installation instructions and the RDocs are somewhat helpful too!

Notes

[1] Rudy has been available on GitHub since the project began, but this release is the first one with documentation.

From WordPress to Jekyll

2009-04-10T00:00:00+00:00

I started this blog in September 2008 with a post about encryption in Ruby. I used Tumblr because I wanted something quick and simple. As it turned out, Tumblr is a great service for personal blogs but not for group and company blogs. Adding multiple users to a single blog with Tumblr is strange. Every user must have their own personal blog before they can be added to the group blog. I also wanted more control over the layout and content.

In December I switched to WordPress. It was a surprisingly pleasant experience. I could do almost everything I wanted to do with the layout and I brought the content home to our own servers. But something still did not feel right. I don’t particularly like databases at the best of times and using one for a blog is more than a bit excessive. There’s a lot of overhead involved in running a WordPress blog (the same thing can be said about any blog engine that combines an application with a database). There’s the system administration to keep it up to date, secure, and performing well. There are more steps than necessary to write new posts. It’s slow. That’s a lot of cake!

So what’s the solution? Jekyll, a blog-aware, static site generator. What’s old is new again. I’ve been using Jekyll for solutious.com for about a month and I’m loving it. Unlike databases, I love files regardless of the times so using a static site is right up my alley. And now I’m using it for the blog too. There’s a lot of positive and helpful stuff written about it already so I won’t re-iterate that here. But I will mention that Jekyll also plays nice with GitHub (which is logical since it’s written by Tom Preston-Werner).

I’ll also mention that I like the name. Thanks TPW!

Installing sys-cpu 0.6.0 on OS X 10.5 Leopard

2008-12-30T00:00:00+00:00

I was trying to install the sys-utils Ruby libraries today and got an error while building sys-cpu on OX 10.5 Leopard. sys-cpu is a multi-platform Ruby library for accessing information about the CPUs and it contains a native extension which needs to be compiled during them installation. This is the error:

$ sudo gem install sys-cpu
  Building native extensions.  This could take a while...
  ERROR:  Error installing sys-cpu:
  ERROR: Failed to build gem native extension.
  ...
  cpu.c:14:17: error: kvm.h: No such file or directory
  cpu.c:14:17: error: kvm.h: No such file or directory
  cpu.c: In function 'cpu_load_avg':
  ...
  make: *** [cpu.o] Error 1

OS X 10.5 seems to be missing a C header file, kvm.h, that’s required by sys-cpu. Here’s how you can solve the problem:

Download a non-gem version and unpack it.
Get a copy of kvm.h. If you’re running OS X 10.5 that was upgraded from 10.4, you can find it in /Developer/SDKs/MacOSX10.4.sdk/usr/include. If not, you can download it from here: http://solutious.com/r/kvm.h.
Copy kvm.h in to sys-cpu-0.6.0/ext
Inside of sys-cpu-0.6.0, run: rake test then sudo rake install
The following command should now run without errors: ruby -r'sys/cpu' -e 1.

That’s it!

From Tumblr to WordPress

2008-12-12T00:00:00+00:00

We’re currently converting our blog from Tumblr to WordPress. We’re literally in the middle of it but I wanted to take a moment to explain why we’re moving and why we chose WordPress.

Why we’re moving from Tumblr

We accepted Tumblr’s strange management for group blogs and the broken archive page for FireFox. But there is one issue that is not excusable: extremely poor search engine performance. This issue was brought up a few months ago by Melissa Chang, a flurry of discussion ensued, and Tumblr made some changes Erik Dafforn posted a more complete chronology). Seven months have passed and Tumblr search engine performance still sucks.

Why WordPress

There’s surprisingly few choices. We looked at some Ruby options Mephisto, enki, Feather, all have strange quirks. We tried them all (and others) because we thought, “Hey it’s been a few years since we installed a blog there must be a new kid on the block”. The shocking answer is, no, there’s not (if you’re looking for something with a reasonable installation process). Mephisto wants Rails 2.2.2. Feather has some issues with Merb and won’t install anyway because gems.datamapper.com seems to be down. Enki just wouldn’t run on Dreamhost (Passenger returned a vague error message but the logs contained nothing helpful). We thought we were on crazy pills but it appears to be reality!

Why WordPress? Because it works.

Update (2008-12-19): WordPress continues to treat us well. I can’t say the same for Disqus however. The wordpress plugin returns broken HTML for the comments_number function (which is why we don’t display the comment count on the left).

Update (2009-04-12): We changed again. The blog is now powered by the static site generator Jekyll. Our comments are now powered by Intense Debate. We finally made it home!

Using Amazon EC2 In Europe

2008-12-10T00:00:00+00:00

If you’re worked with EC2 before, there are a couple things you need to know to get started with launching machine instances in Europe. SSH Keys and AMIs are not shared between regions. We’re going to create both of these and then launch an instance in Europe. There’s also a new parameter --region which you need to use to interact with the new availability zones. Let’s giver’.

##Create an S3 bucket in Europe##

AMIs are stored in S3 so you need a bucket in Europe if you don’t have one already (you’ll probably want a new one for your machine images anyway). There’s a Firefox extension called S3Fox that can help you do this (check the box labeled, “Place this bucket in Europe”). Note that S3Fox is quirky and ugly almost to the point where it’s offensive. But it works and it’s easier than writing a script. The free version of RightScale can do this too. You can also use S3Tools to create buckets but I’ve never used them so I can’t recommend them.

##Copy an AMI from the US to EU##

You have two choices: re-bundle your running instance(s) or copy your existing images with the ec2-migrate-bundle tool. If you choose to re-bundle, follow the steps you normally would (described in the developer guide) and then upload the bundle to Europe (also in the developer guide).

The easiest approach is to use the ec2-migrate-bundle tool. You’ll need to install the latest AMI tools to a Linux or Windows machine near you (these will not work on OSX). As a side note, the AMI tools are written in Ruby which is interesting because the API tools are written in Java.

$ ec2-migrate-bundle --location EU --cert /mnt/cert-*.pem --privatekey /mnt/pk-*.pem \
--access-key ORANGEJUICE --secret-key SUp0rS3kRu7 \
--bucket BUCKET-IN-US --destination-bucket BUCKET-IN-EU \ 
--manifest IMAGE.manifest.xml 

Note: If you’re running this command on an EC2 instance, you’ll need to upload your encryption keys first:

$ scp -i PATH/2/private-key PATH/2/cert-*.pem PATH/2/pk-*.pem root@YOURMACHINE:/mnt/

From your local machine, run the follow to tell EC2 that your new Europe image exists:

$ ec2-register --region eu-west-1 BUCKET-IN-EU/IMAGE.manifest.xml

##Update API Tools##

You won’t be able to see your image yet b/c you need to update your API tools. You should run these on a real, physical machine. Why? Because it’s a good rule of thumb for keeping your keys secure. If you copy them to a machine instance, you could forget them and they could end up being available on every instance you startup.

Now you’ll be able to see your machine images in Europe:

$ ec2-describe-images --region eu-west-1 -o self

You’ll be able to see the regions and availability zones:

$ ec2-describe-regions
REGION	eu-west-1	eu-west-1.ec2.amazonaws.com
REGION	us-east-1	us-east-1.ec2.amazonaws.com

$ ec2-describe-availability-zones --region eu-west-1
AVAILABILITYZONE	eu-west-1a	 available	eu-west-1
AVAILABILITYZONE	eu-west-1b	 available	eu-west-1

You can make Europe your default region by adding the EC2_URL variable to your environment:

# Unix and Unix-like:
export EC2_URL=https://eu-west-1.ec2.amazonaws.com

# Windows:
set EC2_URL=https://eu-west-1.ec2.amazonaws.com

##Create an SSH key in Europe##

Before you can launch an instance in Europe, you need to create a new SSH key:

$ ec2-add-keypair eu-west-1-key --region eu-west-1

The output of this command is your private key. Save it in a secure location! You’ll need to make sure it’s only readable by you (chmod 600 eu-west-1-key). SSH won’t use it otherwise.

##Create a security group in Europe##

The default security group won’t allow you to SSH in to your new instance. You can create a new group with these commands:

$ ec2-add-group --region eu-west-1 -d "Potato Storage" potato
GROUP	potato	Potato Storage

$ ec2-authorize --region eu-west-1 -p 22 potato
GROUP		potato	
PERMISSION		potato	ALLOWS	tcp	22	22	FROM	CIDR	0.0.0.0/0

If security groups are new to you, be sure to check out the documentation and firewall examples. Twenty Rules for Amazon Cloud Security is a good read too.

##Launch an instance in Europe (finally!)##

$ ec2-run-instances --region eu-west-1 --availability-zone eu-west-1b \
--key eu-west-1-key --group potato -t m1.small ami-XXXXXXXX

Use the following command to see when it becomes available:

$ ec2-describe-instances --region eu-west-1

Once it does, log in and have some fun!

$ ssh -i PATH/2/eu-west-1-key root@ec2-XX-XXX-XX-XXX.eu-west-1.compute.amazonaws.com

##Don’t forget to terminate the instance!##

$ ec2-terminate-instances --region eu-west-1 i-XXXXXXXX

##Extra Stuff##

There’s more information about regions in the Amazon EC2 Regions Guide.

Enjoy running your apps in style!

A Basic Voicemail System using Twilio

2008-12-04T00:00:00+00:00

We setup a voicemail system this week using Twilio. There are other similar services but Twilio was the first one we’ve found that’s both available (CloudVox is in private beta) and really simple to get started.

If you aren’t familiar with it, Twilio is a voice communication system with an HTTP/XML API. You create an account with them and they give you a phone number and an API key. Then you write some code that outputs XML (they have libraries for Ruby, Java, Python, and PHP and configure your account to point to an XML resource to handle the initial incoming call. They charge $0.03 or $0.05 cents per minute depending on whether you’re using a local or toll free number.

We’ve posted one of our test configurations to Gist (we blame the PHP on the intern :p sorry Dave). Here’s our initial feedback:

The XML is based on simple verbs (Play, Say, Record…) so it’s easy to get started. I’ve read complaints about using XML in this way, usually on the basis that it’s over simplified and proprietary, but I disagree. If something can be simplified, simplify it. And if it’s easy to work with, the fact that it’s a proprietary language doesn’t matter. (On a side note, is there a standard, open language for controlling voice communication systems?)
The debugger is pretty cool. You can look at the recent requests and responses between their system and your web service. We solved two problems like this (XML encoding issues, again).
The mp3s don’t play properly in Safari (we’re running 3.2.1). It displays a black screen that says “No video”. Sometimes it will play after you hit the spacebar, sometimes not. Firefox is fine. Update: this looks like it could be related to Flip4Mac, which I updated last week. The same thing is happening with other mp3s.
I somewhat concerned about jerks calling in many times or for long durations to charge up our bill. This is an issue with all similar services but I didn’t see anything in the docs or API that would prevent this in Twilio.
The prices are reasonable for moderate use but I expect they should decrease as their customer base increases (they should be able to negotiate lower rates from their suppliers).
Their customer service is excellent! We experienced it and so did our friends at Dealer Diagnostics.

We’ll be looking at other options and we’ll post our findings here.

Sun's EC2 Support sucks

2008-10-17T00:00:00+00:00

So I’m pretty disappointed in Sun. I’ve made good things happen with OpenSolaris and EC2 separately and I was pretty excited about using them together. But we’ve been waiting for three weeks for a reply to my inquiry regarding OpenSolaris I/O performance in Amazon EC2. I can only assume there’s an issue that they’re not ready to talk about because they were very responsive to other questions. This is an unwelcome contrast to the type of support we’re enjoying from Amazon.

My advice: Stick with Linux on EC2.

EC2 Disk Performance, Linux vs OpenSolaris -- Preliminary Results

2008-09-23T00:00:00+00:00

I started playing with the Solaris images on EC2 last week. I’m crazy for disk performance so I got straight to the benchmarking. These are only preliminary results because I’m only testing reading and writing medium to large files and I’m still getting a feel for the performance. In these tests I’m using the dd utility to read and write files to several mounts (local, local swap, and EBS). The test and results are below and I follow up with what will be improved on for the next test. NOTE: I’m currently waiting for a response from Sun’s EC2 support before continuing the tests.

Summary: Reading and writing medium to large files in OpenSolaris in EC2 appears to be slower than Linux. More testing is required.

##Tests##

Read/write 10MB, 100MB, 1000MB to /, /mnt/, /data, /tmp

/ and /mnt are the default “local” disks
/tmp is the default “local” swap space
/data is an EBS volume (ext3 for linux, zfs for solaris)
See script disktest1.sh

##Machines##

m1.small, Fedora 8 Linux 32-bit (test data)
m1.small, OpenSolaris 2008.05 32-bit (no test data, killed after 20 minutes)
m1.large, Fedora 8 Linux 64-bit (test data)
m1.large, Solaris Express Community Edition build 79 64-bit (test data)

##Conclusions##

Local Disk performance: Linux is much faster.

Linux vs OpenSolaris
OpenSolaris can be up to 30 times slower for 1GB+ files (2s vs 1m)
About the same for 10MB and 100MB files (~0.02s)
OpenSolaris swap (/tmp) performance is about 2x slower (2s vs 4s for 1GB)

Small Linux vs Large Linux

Comparable for 10MB files
m1.small is amall is about 3x slower for 100MB files (0.6s vs 0.2s)
m1.small is amall is about 5-8x slower for 1000MB files (12s vs 2s)

Small OpenSolaris vs Large OpenSolaris

The m1.small instance of OpenSolaris is really, really slow for all tests. I stopped the test after 20 minutes (note the m1.large OpenSolaris instance took 6.5 minutes)

EBS Performance: Linux is faster.

Linux vs OpenSolaris
Solaris performance is about 2.5x slower (2s vs 5s for 1GB)
Small Linux vs Large Linux
m1.small is about 2x slower for 10MB (0.05s vs 0.025s)
m1.small is about 2-3x slower for 100MB (0.6s vs 0.2s)
m1.small is about 7x slower for 1GB (14s vs 2s)
Small Solaris vs Large Solaris
m1.small is REALLY slow.

##Notes##

The times grew longer for each of the 3 runs (the first run was always the fastest). Try again with a delay in between to account for some kind of background EC2/EBS process to finish.
There are occasional writes which are very slow for both platforms regardless of which test was being run.
The Solaris Express Community Edition is not supported.

##What you can look forward to in the next test##

Use standard public EC2 images. Include the ami ID and uname output for each test to be clear in the systems used.
Track the total run time for all tests (note: above I only have times for the two large instances).
Use bonnie-64 test to test caching / swapping performance.
Use bonnie++ for testing small file performance.
Restart instances between tests, define warm-up procedure.

Simple Encryption (in Ruby)

2008-09-16T00:00:00+00:00

I occasionally get questions on how we do authentication in Ruby so I thought I’d write this post so I could direct people here. A lot of the authentication stuff we do is based on this simple implementation of RSA key support. There were a couple minor syntax errors in their crypto-key.rb though so here’s a fixed version:

require 'openssl' 
require 'base64'	# Added 

module Crypto
  
  def self.create_keys(priv = "rsa_key", pub = "#{priv}.pub", bits = 1024)
    private_key = OpenSSL::PKey::RSA.new(bits)
    File.open(priv, "w+") { |fp| fp << private_key.to_s }
    File.open(pub,  "w+") { |fp| fp << private_key.public_key.to_s }    
    private_key
  end
  
  class Key
    def initialize(data)
      @public = (data =~ /^-----BEGIN (RSA|DSA) PRIVATE KEY-----$/).nil?
      @key = OpenSSL::PKey::RSA.new(data)
    end
  
    def self.from_file(filename)    
      self.new File.read( filename )
    end
  
    def encrypt(text)
      Base64.encode64(@key.send("#{key_type}_encrypt", text))
    end
    
    def decrypt(text)
      @key.send("#{key_type}_decrypt", Base64.decode64(text))
    end
  
 def private?()  !@public; end # Added () and ;

 def public?()   @public;  end # Added () and ;
    
    def key_type
      @public ? :public : :private
    end
  end
end