I’m not excepting myself, either. I see myself doing this all the time. “It’ll make things simple”, I say. Which by a strict interpretation is true. Whatever direction I’m advocating, it usually makes [some] things simple. It usually also makes other things more complicated.

Let’s look at an example of two opposing kinds of simplicity. My old boss Chris Strom recently wrote about some common Ruby newbie mistakes (it’s a great article, you should read the whole thing). Here is an example he gave of code written by someone new to the Ruby language:

sum = 0
i = 0
while i < times.length
  time = times[i]
  # parse / manipulate the time
  sum = sum + time
  i = i + 1
end

After several iterations of successively introducing Ruby idioms and features, the code looked like this:

def average_time_of_day(times)
  sum = times.map(&:to_time).inject(&:+)
end

Which example is simpler? Well, that depends on what you mean by "simple". As Chris points out, the latter example is much closer to the domain. The line sum = times.map(&:to_time).inject(&:+) neatly expresses the intent of the code without getting bogged down too much in implementation. It's also more concise.

On the other hand, the first example uses language constructs that are familiar to almost all programmers, not just Ruby programmers. It is simpler in the sense that even someone who has only basic programming skills could probably work their way through it, without needing to understand concepts like #inject and symbol-to-proc.

What do we mean when we say the word "simple"? I've realised that we programmers use "simple" to mean a lot of different things. Some of the ways we use the word "simple" include:

Minimizing unnecessary effort. Characterised by the classic admonition to "Keep It Simple, Stupid". Or as the XP folks say, "do the simplest thing that could possibly work". Of course, sometimes when you choose to do less you force someone else to do more. And sometimes that someone else is you, two weeks down the road.

Hiding complexity. Ruby on Rails made web development simple, relative to tools that came before it. It accomplished this feat by embracing "convention over configuration". In order to do this, it had to incorporate a great deal of extra complexity in the form of algorithms which guess the intent of the programmer instead of forcing her to specify her wishes explicitly. Anyone using it eventually has to become familiar with these hidden rules in order to understand why Rails behaves the way it does.

Avoiding difficult-to-understand features. In the name of simplicity, some programmers advocate avoiding "magic" language features they see as complicated and difficult, such as generics, recursion, or metaprogramming. Other coders say there is no such thing as magic if you have an adequate understanding of the language, and that arbitrarily naming certain features "magical" is superstitious nonsense.

Avoiding formal architecture. There are programmers who feel that complexity is the inevitable result of spending too much time thinking about design, and that simple code is code that does it's job without being fastidiously decoupled and thoughtfully abstracted. Others say that this kind of thinking leads to an unmaintainable "big ball of mud", the antithesis of simplicity.

Elegance of design. Some insist that elegant orthoganality is what defines simplicity. Simple components which do one thing well, which have few interdependencies and can be composed into various configurations. Others retort that all abstractions are leaky and that trying to abstract the leaks away only complicates the job of solving real problems.

Staying close to the domain. To some, simplicity is ability to write code which looks like a plain-English explanation of the problem. But the underlying scaffolding which makes the code so understandable to a domain expert may be baroque and brittle in it's implementation.

Clearly, simplicity is not a simple subject.

I think the reason for this is the nature of complexity. Complexity is like an air pocket trapped under a layer of airtight plastic. If you push it down it just pops up somewhere else. Most of our attempts to "simplify" software really amount to pushing the complexity somewhere (or some-when) else. And our disagreements about what is meant by simplicity are really about where we think the complexity belongs. As a result, the goal of simplicity can be used to justify just about any course of action.

I challenge myself and anyone who reads this, next time you use the word "simple", to stop and think about what you mean by it. As an exercise, try to rephrase it in terms of where you are pushing the complexity. Let me know if you gain any insight from looking at it in this way.

  require 'progressbar'
  require 'right_aws'

  def sdb_copy(source_domain, destination_domain)
    sdb = RightAws::SdbInterface.new
    sdb.list_domains do |results|
      domains = results[:domains]
      if domains.include?(destination_domain)
        raise "Destination #{destination_domain} already exists"
      end
    end
    sdb.create_domain(destination_domain)
    count = session.select("select count(*) from #{domain}")[:items].first["Domain"]["Count"].first.to_i
    progress = ProgressBar.new('copy', count)
    sdb.select("select * from #{source_domain}") do |results|
      results[:items].each do |record|
        record.each_pair do |key, data|
          sdb.put_attributes(destination_domain, key, data)
          progress.inc
        end
      end
    end
    progress.finish
  end

This method has a few bells and whistles: it checks to see that the destination domain doesn’t already exist, and it displays a progress bar to let you know how far along the copy process is.

Unfortunately since SimpleDB doesn’t support any kind of bulk import the copies have to be made one by one, which is slooooow. A copy of about 45,000 items takes over 2 hours.

I’ve posted a Gist with this method and a few other utilities for interacting with SimpleDB. It’s still in a pretty rough state, but it’s usable. If you use Boson you should be able to install it as a boson library with the following command:

$ boson install http://gist.github.com/raw/220299/171c2b3c8aae955afa1dc38a78c77c0531e24d1c/simpledb.rb

Lately I’ve been working with DataMapper and I’d begun to wonder how difficult it would be to write a DataMapper version of NullDB. As it turns out, the answer is “trivially easy”. In fact, it probably doesn’t even warrant making a library out of it – all the tools needed are right in the DataMapper distribution.

The task of a DataMapper null adapter is simplified in comparison to the ActiveRecord version by dint of DataMapper’s differing architecture. Where ActiveRecord scans the DB schema to extrapolate tables and columns, DataMapper makes the model classes the sole canonical source of schema information. This means that all the mucking about with schema.rb files that the original NullDB had to do is unnecessary.

The simplest form of a null database adapter is one that simply raises an exception every time code tries to touch the database. This kind of null backend is useful when you want to ensure that the software under test is completely isolated from the database. Rails Views are one example of code which you might want to keep rigidly database-independent in this fashion.

Setting up this type of null database backend in DataMapper is trivial:

  DataMapper.setup(:default, "abstract::")

This code uses the “abstract” adapter which comes with DataMapper. The Abstract adapter is primarily intended to provide a base class on which to build other adapters. But for our purposes it makes for a poor-man’s null database backend. Any attempt to write to or query from the database while this adapter is in effect will result in a NotImplementedError being raised:

  p = Post.new(:title => "NullDB for DataMapper")
  p.save # => NotImplementedError

What if we want to incoporate this technique into a complete test suite which includes both database-dependent and database-free tests? We need a way to make the null database active only while selected tests are being run. Once again, DataMapper’s design makes it easy to achieve our ends. DataMapper incorporates the concept of a “repository stack”. instead of only having a single database adapter active for a whole session, or having different models tied to different adapters, DataMapper keeps a stack of adapter sessions in a stack. New sessions can be pushed onto the stack, and then popped off when they are no longer needed.

Here’s an example of how to set up a single Test::Unit test case to use a null database backend, while leaving other tests unaffected:

class FooTest < Test::Unit::TestCase
  def setup
    DataMapper.setup(:null, "abstract::")
    DataMapper::Repository.context.push(DataMapper.repository(:null))
  end

  def teardown
    DataMapper::Repository.context.pop
  end

  def test_bar
    # ...
  end
end

There’s one kind of test that NullDB enables which the code we’ve looked at so far does not allow. That is, testing of after-save hooks. We can’t test an after-save hook with the code above, because a call to #save will error out before it ever gets to the after-save callback.

While DataMapper does not include an actual null-object style backend which turns all database operations into no-ops, it includes something close which will permit us to test after-save hooks. DataMapper ships with an in-memory store, which implements the DataMapper API in terms of a simple in-memory collection of Ruby objects. Here’s an example of triggering an after-save hook with an in-memory backend:

class Foo
  include DataMapper::Resource

  property :id, Serial

  after :save do
    puts "Saved!"
  end
end

DataMapper.setup(:ephemeral, "in_memory::")

DataMapper.repository(:ephemeral) do
  f = Foo.new
  f.save
end

Running this script produces the output Saved!. This code also demonstrates the block-form of the the DataMapper.repository method, which pushes a session onto the repository stack, yields to the passed block, and then pops the session back off the stack.

So there you have it, a couple of quick and easy ways to approximate a null-database in DataMapper. I think it's a testament to DataMapper's design that it's so easy to assemble the pieces it provides in ways that the writers may not have anticipated. I hope you find these techniques useful!

• Unambiguous. There is never any question how to interpret them.
• Human-readable. You can look at dates stored in ISO8601 and interpret them easily.
• Widely-supported. All major programming languages have libraries for parsing and writing ISO8601 dates.
• Sortable. If you follow a few simple rules, ISO8601 dates sort lexicographically into the order you would expect.

That last property makes ISO8601 a particularly good candidate for date/time storage in simple non-relational data stores like Amazon’s SimpleDB. Storing dates as ISO8601 lets you sort records by date in data stores which don’t support a native date/time data type – without sacrificing human-readability.

Using ISO8601 from Ruby couldn’t be simpler, if you know where to look for the tools. If you look at the Ruby Time class by itself, you won’t find any reference to ISO8601. The secret is to require the ‘time’ library:

irb(main):001:0> time = Time.now.utc.iso8601
NoMethodError: undefined method `iso8601' for Mon Oct 26 04:46:57 UTC 2009:Time
        from (irb):1
        from :0
irb(main):002:0> require 'time'
=> true
irb(main):003:0> time = Time.now.utc.iso8601
=> "2009-10-26T04:47:09Z"
irb(main):004:0> Time.iso8601(time)
=> Mon Oct 26 04:47:09 UTC 2009

Note the use of utc to convert the local time into universal time (aka Greenwich Mean). This is important if you want your time strings to be lexicographically sortable. The “Z” on the end of the ISO8601 date indicates that it is in UTC, not a local time zone.

#ifndef HELLO_H
#define HELLO_H

/* ... C code ... */

#endif

At first the situation in Ruby seems much improved. We have require, after all, which ensures that a given file will only be loaded once. Or does it?

As it turns out all is not rosy in Ruby-land. Require works great for gems and system libraries. But when we start loading files relative to the current file, the old double-load problem rears it’s ugly head once more.

Let’s take a look at why this happens. First, we’ll define a file to load:

# foo.rb
class Foo
  puts "I'm being loaded!"
end

Now we’ll define a client file which requires foo.rb:

# client.rb
require File.join(File.dirname(__FILE__), './foo')
require File.join(File.dirname(__FILE__), '../lib/foo')

When we run the script above, we get the following output:

I'm being loaded!
I'm being loaded!

Of course, we’d never write a file like that, with the same library being required twice using two different paths. But in larger projects it is all too common for a file to be required using a different path in different files. Because Ruby does not use canonicalized pathnames to check if it has already loaded a file, it assumes that the different paths must refer to different files and loads the file over and over again.

Is this a problem? Besides for slower application startup, the most common ill effect of repeated file loads is constant redefinition warnings. If you have a project that outputs a lot of warnings that look like this on startup…

../lib/foo.rb:1: warning: already initialized constant FOO

…you probably have some files being loaded twice or more times.

More serious and subtle side-effects of double-loading can occur though, especially if the files being reloaded do any class-level metaprogramming. Errors caused by double-loading can be strange and very difficult to track down.

So what do do? Well, first we need to find where the offending loads are originating from. In a large project this can be a daunting task. Here’s some code I wrote to help track down double loads at Devver:

ROOT_PATH = File.expand_path('..', File.dirname(__FILE__))

def require_with_reload_check(raw_path)

  unless $LOADED_FEATURES.include?(raw_path)
    $require_sites ||= {}
    site, line, info = caller.first.split(':')
    expanded_site = File.expand_path(site)
    load_dir = $LOAD_PATH.detect{|dir|
      File.exist?(File.expand_path(raw_path + ".rb", dir))
    }
    expanded_path = File.expand_path(raw_path, load_dir)

    if (expanded_path.index(ROOT_PATH) == 0) &&
        $require_sites.key?(expanded_path) &&
        $require_sites[expanded_path][:as] != raw_path &&
        expanded_path !~ /test_helper$/
      warn "!" * 80
      warn "#{expanded_path} is being reloaded!"
      warn "It was originally loaded as: #{$require_sites[expanded_path][:as]}"
      warn "From #{$require_sites[expanded_path][:in]}"
      warn "But now it is being loaded as: #{raw_path}"
      warn "In #{expanded_site}"
      warn "!" * 80
    end

    $require_sites[expanded_path] = {
      :as => raw_path,
      :in => expanded_site
    }
  end
end

unless defined?($reload_guard_enabled)
  alias require_without_reload_check require
  alias require require_with_reload_check
  $reload_guard_enabled = true
end

This code should be loaded as early as possible in your project. Once loaded, it spits out some a very noisy warning every time a file is re-loaded using a different path:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
/home/avdi/articles/double-load-guards/lib/foo is being reloaded!
It was originally loaded as: ././foo
From /home/avdi/articles/double-load-guards/lib/client.rb
But now it is being loaded as: ./../lib/foo
In /home/avdi/articles/double-load-guards/lib/client.rb
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

But how do we avoid reloads, once we have located the offenders? The simplest remedy is to always expand relative paths before requiring them. I prefer to use the two-argument form of File.expand() to construct fully-qualified paths:

# client.rb
require File.expand_path('../lib/foo', File.dirname(__FILE__))

Eliminate your double-loads, and your Ruby code will load faster, produce fewer warnings, and be that much less prone to bugs.

There’s a problem with the Ruby integration, though, which is that the temporary files it generates frequently get mistaken for project files by tools like Autotest and Merb. I fixed this in my .emacs a long time ago, and after the second person asked about how I fixed it I thought I’d post my fix publicly.

Note: the following is a horrible kludge and the only nice thing I can say about it is that it works. In a perfect world Flymake would make it possible to customise the directory it uses instead of insisting on creating it’s files in the same directory as the originals. My version is monkeypatching in the first degree and if someone would take the time to clean it up and make an honest patch of it I’d happily use it.

As I’ve mentioned in the past I hate null checks. They clutter up code and add nothing meaningful to the code’s narrative. I hate checks to see if a value is singular or plural too. I guess what I’m saying is that I hate type checking and avoid it wherever possible.

Here’s a contrived example of some code that does both a null check and a cardinality check. It takes an argument and produces some HTML list item tags:

def make_list(items)
  case items
  when Array then items.map{|item| html "<li>#{item}</li>" }.join
  when nil then ""
  else "<li>#{items}</li>" # singular value
  end
end

Blech. Thankfully there is an elegant idiom which cleans this code up nicely:

def make_list(items)
  Array(items).map{|item| "<li>#{item}</li>" }.join
end

Kernel.Array() is the “Array-ifier” method. It takes a value and always returns an array. And it’s pretty smart about how it translates:

Array("foo") # => ["foo"]
Array(nil) # => []
Array([1,2,3]) # => [1,2,3]
Array(4..6) # => [4,5,6]
Array({:a => 1, :b => 2}) # => [[:b, 2], [:a, 1]]

As you can see, whatever you give it, you always get an Array back. Note in particular that if it is passed an Array, it returns the argument unaltered rather than wrapping it in another layer of array.

Under the covers, Array() tries to call #to_ary on the passed object, and if that fails it tries #to_a.

If you’ve been using Ruby for a while you might object “but isn’t that what #to_a is for?”. The problem with #to_a is that it’s old default behavior, which was to wrap a non-Array object in an Array, is deprecated. Array() is the future-proof way to array-ify arbitrary objects.

Now go forth, and write self-confident code!

Here ya go:

require 'tempfile'

def browse_string(body)
  Tempfile.open('browse_string') do |f|
    browser = ENV.fetch('BROWSER') { 'firefox' }
    f.write(body)
    f.close
    puts "Starting browser #{browser} with response..."
    unless system(browser, f.path)
      warn "Error starting browser #{browser}"
    end
  end
end

Just drop that in your debug script, in your .irbrc, or make a Boson command out of it. Then call it with the HTML you want to view:

browse_string(response.body)

It even respects the BROWSER environment convention (well, sort of).

I’ve put the code on Gist as well.

category = 'riddles'
fortune = `fortune #{category}`

Q:      What's the difference between a Mac and an Etch-a-Sketch?
A:      You don't have to shake the Mac to clear the screen.

It all seems fine and good until some wise-ass comes along and messes things up for everyone.

category = '; rm -rf *'
fortune = `fortune #{category}`

It doesn’t have to be this way though. You can force process arguments to be interpreted strictly as strings instead of being subject to shell evaluation. You just have to use the multi-argument form of methods which start subprocesses. This means that backquotes are out. We can use #system() though…

filename = '; rm -rf *'
system('test', '-e', filename) # => false

That doesn’t help us with our original example, though. We need the output. Open3 will do the trick:

require 'open3'
category = '; rm -rf *'
output = ""
fortune = Open3.popen3('fortune', category) do |stdin, stdout, stderr|
  output += stdout.read
  output += stderr.read
end
puts output

No fortunes found

This technique is not a sure-fire defence against malicious data. It will be of little help if the command you are invoking can start a subshell, so be especially careful when calling out to shells or to other scripting language interpreters. But for simple commands it can provide a layer of protection.

I thought I’d post the Makefile I’ve been creating as I work on my first WebOS app. It’s definitely a work in progress, but it’s capable of building packages, launching the emulator, launching novacomd as a background job, and reloading the app. I’ve tried to get the dependencies right so that altering any part of the app forces a package rebuild before installing.

Note, this makefile is for Webos SDK 1.2 on an Ubuntu system. It will probably require some small modifications to work on an OS X system.

The Makefile, along with a small helper script for launching novacomd, can be found at http://gist.github.com/212448. Please let me know if you find it useful!

EDIT: I realised there’s one part of the Makefile which might not be immediately clear. In order to keep things DRY, I have the Makefile be the sole source of the app version (in the VERSION variable). Then I generate the appinfo.json file from a template (appinfo.json.m4) using the GNU M4 macro processor and passing in the app version.