You get the idea.

I highlighted what I think is the key point n this conversation: the fact that the Skeptic learned about Practice X from a book. It could just as well have been a Wikipedia article or a series of blog posts. The point is that our Skeptic tried to put into practice something which he had never witnessed first-hand, and was disappointed at the results.

It strikes me that there is a form of cargo-culting going on here. Only in this case, when the cargo planes fail to arrive, the cultists experience an understandable crisis of faith. Sometimes followed by a period of angry backlash against the”faithful”.

In some Buddhist traditions the concept of “Dharma transmission” is held to be central. A Dharma transmission is the direct person-to-person teaching from master to student. A teaching is regarded as true and authentic if a lineage of direct transmissions can be traced, teacher to teacher, all the way back to Sakyamuni Buddha himself.

Dharma transmission cannot occur through reading holy books. It must occur in person. I think this may sometimes be true of software development practices as well. We have a wealth of books and articles on best practices. You can learn everything there is to know about Test-Driven Development or Separation of Concerns without ever leaving your desk. But without sitting down with a master and working through a real-world problem, you risk missing that moment of enlightenment when everything clicks into place.

I’m usually quick to say “you’re doing it wrong; read more and try again” when I see people complaining about practices that I find profoundly beneficial. But maybe that isn’t helpful. Maybe some understandings can only be effectively passed by direct transmission.

For instance, here’s a recursive reimplementation of Ruby’s Array#join method which splits the joined list into first and rest:

def rec_join(list, delim)
  case list.size
  when 0 then ""
  when 1 then list.first.to_s
  else list.first.to_s + delim + rec_join(list[1..-1], delim)
  end
end

rec_join([:foo, :bar, :baz], ', ') # => "foo, bar, baz"

In this example we used list.first to get the first element, and list[1..-1] to get the rest. But is this a robust way to split an array into head and tail components? Let’s see:

zero = []
one  = [0]
many = [0,1,2,3,4]

first, rest = zero.first, zero[1..-1]
first                           # => nil
rest                            # => nil
first, rest = one.first, one[1..-1]
first                           # => 0
rest                            # => []
first, rest = many.first, many[1..-1]
first                           # => 0
rest                            # => [1, 2, 3, 4]

Hmmm. We said that rest should always be a list with zero or most elements; but when given an empty list to start with, this technique gives us nil as the tail of the list. It’s also a bit ugly. Let’s see if we can do better.

first, rest = *zero
first                           # => nil
rest                            # => nil
first, rest = *one    
first                           # => 0
rest                            # => nil
first, rest = *many        
first                           # => 0
rest                            # => 1

Now we’re using the splat operator to break the source list into it’s parts. The code is much more concise, but the results are very wrong. In this version rest is never a list!

However, a slight refinement and we have exactly what we want:

first, *rest = *zero
first                           # => nil
rest                            # => []
first, *rest = *one    
first                           # => 0
rest                            # => []
first, *rest = *many        
first                           # => 0
rest                            # => [1, 2, 3, 4]

Here we’ve added a splat operator to the left side of the variable assignment as well, telling Ruby to collect all remaining values into an Array assigned to the rest variable. This version works exactly as we intended, and is still concise. I first saw this technique used in the DataMapper codebase, and it’s the most elegant way I’ve come across to break a Ruby Enumerable object into first and rest components.

1. Stand still!
2. The audience is in front of you, dumbass!

The slides are available here. I used GNU Emacs and Org-Mode to write the presentation and all of the code samples, org-s5.el to convert the Org file into an S5 presentation, and the GIMP for the source code overlays.

Ruby users are not so lucky. Our first indication of an error is either the code not working or a stack trace, depending on whether the error is handled somewhere. Either way, we don’t get an opportunity to investigate the circumstances of the error when it is raised. All we get to see is the aftermath.

Ruby exceptions are raised with the raise method, or with fail, which is an alias for raise. I use the term “method” deliberately. raise is not a keyword; it’s a method on Kernel just like puts and exit.

The fact that raise is just an ordinary method has some interesting implications. For instance, we can modify it to trace the location of every error raised in a program:

module MyRaise
  def raise(*args)
    puts "Error at #{caller.first}!"
    super(*args)
  end
end

class Object
  include MyRaise
end

raise "blah" rescue nil         # => nil
# >> Error at -:12!

Since MyRaise is included in Object after the default Kernel, our definition of raise takes precedence.

Obviously, if we can intercept errors at the moment they are raised, there’s a lot more we can do beyond mere tracing. Introducing Hammertime.

Hammertime is an interactive error console for Ruby. Using it is simple: just gem install hammertime and require the library:

require 'hammertime'

Now when an error is raised, we’re be presented with a menu:

=== Stop! Hammertime. ===
An error has occurred at example.rb:6:in `faulty_method'
The error is: #<RuntimeError : Oh no!>
1. Continue (process the exception normally)            
2. Ignore (proceed without raising an exception)        
3. Permit by type (don't ask about future errors of this type)
4. Permit by line (don't ask about future errors raised from this point)
5. Backtrace (show the call stack leading up to the error)              
6. Debug (start a debugger)                                             
7. Console (start an IRB session)                                       
What now?

With Hammertime we can diagnose and fix errors at the point where they occur. Let’s walk through an example Hammertime session using the following highly contrived script:

$broken = true

def faulty_method
  raise "Oh no!" if $broken
end

3.times do |n|
  puts "Attempt (#{n+1}/3)"
  begin
    faulty_method
    puts "No error raised"
  rescue => error
    puts "Error raised: #{error.inspect}"
  end
end

We start the code with Hammertime enabled:

$ ruby -rhammertime example.rb

The code raises an error, and we see the Hammertime menu:

Attempt (1/3)

=== Stop! Hammertime. ===
An error has occurred at example.rb:6:in `faulty_method'
The error is: #<runtimeerror : Oh no!>
1. Continue (process the exception normally)
2. Ignore (proceed without raising an exception)
3. Permit by type (don't ask about future errors of this type)
4. Permit by line (don't ask about future errors raised from this point)
5. Backtrace (show the call stack leading up to the error)
6. Debug (start a debugger)
7. Console (start an IRB session)
What now?

We choose “Backtrace” to see the full trace leading up to the error:

5
example.rb:6:in `faulty_method'
example.rb:12
example.rb:9:in `times'
example.rb:9
1. Continue (process the exception normally)
2. Ignore (proceed without raising an exception)
3. Permit by type (don't ask about future errors of this type)
4. Permit by line (don't ask about future errors raised from this point)
5. Backtrace (show the call stack leading up to the error)
6. Debug (start a debugger)
7. Console (start an IRB session)
What now?

Note that Hammertime returns us to the menu after showing the stack trace. Next we choose “Console” to drop into an IRB session. From there, we do a little snooping around:

>> $broken
=> true

Well that seems to be the problem, someone left the “more magic” switch off. Let’s fix that:

>> $broken=false
=> false
>> exit
1. Continue (process the exception normally)
2. Ignore (proceed without raising an exception)
3. Permit by type (don't ask about future errors of this type)
4. Permit by line (don't ask about future errors raised from this point)
5. Backtrace (show the call stack leading up to the error)
6. Debug (start a debugger)
7. Console (start an IRB session)
What now?

And now we’ll choose to Ignore the error and proceed:

What now?
2
No error raised
Attempt (2/3)
No error raised
Attempt (3/3)
No error raised

We’ve fixed the “bug”, so successive runs execute without problems.

Hammertime has other features, including the ability to specify certain errors which won’t trigger the menu to be presented. If you want to learn more about it I suggest installing it, playing around with it, and reading the code.

The biggest limitation of Hammertime is that it can only catch errors which are raised within Ruby code, not those raised in native code. I threw it together in an afternoon, so it probably has other bugs as well. Please let me know if you get some use out of the tool. Patches, bug reports, and suggestions are welcome.

UPDATE: Magnus Holm was inspired by this post to create an Exception#continue method. With a little more work I think this could be the basis for a Lisp-style conditions system, something I’d love to see implemented in Ruby.

Some may find this easter egg as whimisical as its authors apparently intended. Personally, I find it to be an unprofessional abuse of my IM credentials and identity. When we as users turn our logins and passwords over to a social software client, we implicitly trust that software to safeguard our our identity. There is a reason people get irate when a Twitter app spams all their friends with messages without their express permission. Impersonating a user and spamming their friends is a violation of trust.

I think every programmer goes through stages of being enamored with certain practices. Just like many other coders after I first read Design Patterns I looked for excuses to use patterns everywhere. After becoming “test-infected” I wouldn’t write code without tests up-front – even if it meant spending a lot of time constructing elaborate test harnesses. And then later on, just like many programmers before me, I moved on to a point where these practices were no longer ideologies but merely tools in my toolbox.

As programmers and engineers we are obsessed with optimizing processes, and I think for many of us after we “level up” our skill we look for ways the learning process could have been optimized. A lot of developers, when moving past strict rule-based approaches to coding adopt the idea that all those rigid procedures are really for the stupid mediocre programmers; here among the 10% we don’t need that sort of thing. But I don’t think it’s that simple.

There are things you learn from consistent practice of a discipline which no amount of reading about the discipline can instill. Learning to view software in terms of patterns and antipatterns improved my skill. So did learning to illustrate software in UML. So has practicing strict TDD. So has practicing pair-programming. The agile masters will tell you to practice one of the Agile methodologies fully and strictly before trying to customize it for your organization. There is intangible value in discipline.

I’m all for embracing simplicity (so long as we’re clear about what we mean by it). But I think we should recognize that disciplines, while not the end goal, are also not necessarily obstacles to ahieving mastery. Rather, they are essential stepping stones.

It’s your fault.

I have GMail set up to SMS me when I break the build. Nearly every time I get the dreaded notification, I go through the same defensive routine. “But the tests passed locally!” I protest. “I wasn’t working on anything that would have affected that!” I explain. “There’s no way that could POSSIBLY be related” I whine. “That shouldn’t even be possible!” I complain. “It’s not my fault!”

Nine times out of ten, though, after grudgingly debugging the issue I discover that yes, in fact, my changes were the culprit. Some unforseen side-effect of my changes caused the full-stack tests to fail. It’s usually an easy fix, albeit not always an obvious one.

The lesson here? Software is complex, and my brain is small. I often don’t even remember all the changes that went into a particular feature commit; there’s no way I can predict every tertiary effect those changes will have. Nine times out of ten I’d save myself some time and my team some grief if I just assumed it was my fault and got to work.

warning: peer certificate won't be verified in this SSL session

No one likes warnings cluttering up their output, and this one indicates a very real security problem: no verification is being done to check that the server really is a legitimate Amazon host. Unfortunately, the warning doesn’t give you a lot of clues about how to go about addressing the issue.

Here’s what you need to do. First, you’ll need a certificate file. If you’re running Ubuntu, you can install the “ca-certificates” package (on my system it was already installed). Once it’s installed you should have a master certificate file at /etc/ssl/certs/ca-certificates.crt. Now you just need to tell RightAWS about the file:

require 'right_http_connection'
Rightscale::HttpConnection.params[:ca_file] = "/etc/ssl/certs/ca-certificates.crt"

…and you should get no more warnings.

I’ve submitted a patch to the right_http_connection gem which, if accepted, will use this file by default if it exists. It will also enable users to override the CA file configuration using the RIGHT_HTTP_CA_FILE environment variable.

select * from my_domain where date > '2009-01-01'

The only documented way to retrieve the attributes of an item by item name is with the GetAttributes operation, which only gets one item at a time. But what if you have a list of item names you want to retrieve, and you just want to avoid the overhead of making a separate GetAttributes call for each one?

A clue towards the solution can be found in the fact that the special itemName() token can be used in the field list for Select:

select itemName() from my_domain where date > '2009-01-01'

select * from my_domain where itemName() = 'ABC123'

…and from there it’s a short step to our solution, which uses the IN predicate:

select * from my_domain where itemName() in ('ABC123', 'DEF456', ...)

And there you have it, a way to retrieve multiple items by name in a single request, subject only to limits on query size and response size. I don’t know if this is common knowledge to longtime SimpleDB users, but it was new to me, so I thought I’d share it.