Alex Rothenberg

Programming With Kids

2011-11-14T00:00:00-08:00

I’ve started to teach my kids to program. I figured I build websites professionally and it’d be a fun way for me to share what I do and help supplement their learning. And it was something they expressed interest in not something I was pushing. I suggested we build our own small version of facebook or twitter. Very quickly I learned two truths

websites are boring
games are fun

Okay. I’ve never built a game before after a little digging there are plenty of tools in the open source world and many built on Ruby. We’re currently experimenting with three different tools/technologies.

Shoes

I first came across Shoes several years ago and was blown away. It was originally written by why and is now maintained by Team Shoes on github. “Shoes is the best little DSL for cross-platform GUI programming there is. It feels like real Ruby, rather than just another C++ library wrapper”

Writing a Shoes app feels just like writing a Ruby app (it is Ruby!). The best analogy I can use is that what Rails does for websites, Shoes does for GUI apps.

If you want to create a blue rectangle on a page, here’s your app

Shoes.app do
  fill blue
  rect :top => 25, :left => 50, :height => 75, :width => 150
end

We can make it a bit more interactive and allow the user to move our rectangle around with the arrow keys and display the current coordinates

Shoes.app do
  fill blue
  @player = rect :top => 25, :left => 50, :height => 75, :width => 150
  @current_coordinates = para "(#{@player.left}, #{@player.top})"

  keypress do |key|
    @player.left += 10 if key == :right
    @player.left -= 10 if key == :left
    @player.top  += 10 if key == :down
    @player.top  -= 10 if key == :up
    @current_coordinates.replace "(#{@player.left}, #{@player.top})"
  end
end

We’re not limited to blue rectangles. We can replace it with an image

Shoes.app do
  @player = image 'images/Starfighter.png', :top => 25, :left => 50
  @current_coordinates = para "(#{@player.left}, #{@player.top})"

  keypress do |key|
    @player.left += 10 if key == :right
    @player.left -= 10 if key == :left
    @player.top  += 10 if key == :down
    @player.top  -= 10 if key == :up
    @current_coordinates.replace "(#{@player.left}, #{@player.top})"
  end
end

We’re writing Ruby in a fairly natural way. Shoes just gives us GUI methods like rect to create a rectangle or para to create a paragraph. Because this is Ruby, as your Shoes app gets more complex you can create classes and methods to organize and keep it manageable just as you would in any other app.

There are all sorts of great resources out there including

The Shoes Manual which includes a reference for all the elements you may use (like rect or para)
why’s tutorial - the original introduction to Shoes
sample apps - there are some really good ones here!
Teaching Ruby to High School Girls (using Shoes) article by Sarah Mei

Gosu

Gosu is a gaming library so while Shoes lets us build any sort of GUI apps this is seemed like it might be a better fit since we’re interested in gaming. Luckily there’s a gem that wraps up the Ruby interface (Gosu can be used from C++ or Ruby).

gem install gosu

If we want to build a similar game to what we did in Shoes with a play we move around via the arrow keys we need to subclass the Gosu::Window class.

require 'rubygems'
require 'gosu'

class Player
  def initialize(window)
    @image = Gosu::Image.new(window, "media/Starfighter.png", false)
    @x, @y = 125, 50
    @angle = 0.0
  end

  def draw
    @image.draw_rot(@x, @y, 0, @angle)
  end
end

class GameWindow < Gosu::Window
  def initialize
    super(640, 480, false)
    self.caption = "Gosu Tutorial Game"
    @player = Player.new(self)
  end

  def draw
    @player.draw
  end
end

window = GameWindow.new
window.show

And we see a window with a player that looks like a starship

Its not too hard to make it move by overriding the update method in our window class

require 'rubygems'
require 'gosu'

class Player
  attr_accessor :x, :y

  def initialize(window)
    @image = Gosu::Image.new(window, "media/Starfighter.bmp", false)
    @x, @y = 75, 50
    @angle = 0.0
  end

  def draw
    @image.draw_rot(@x, @y, 0, @angle)
  end
end

class GameWindow < Gosu::Window
  def initialize
    super(400, 300, false)
    self.caption = "Our Game"
    @player = Player.new(self)
    @current_coordinates = Gosu::Font.new(self, Gosu::default_font_name, 20)
  end

  def update
    @player.x -= 10 if button_down? Gosu::KbLeft
    @player.x += 10 if button_down? Gosu::KbRight
    @player.y += 10 if button_down? Gosu::KbDown
    @player.y -= 10 if button_down? Gosu::KbUp
  end

  def draw
    @player.draw
    @current_coordinates.draw("(#{@player.x}, #{@player.y})", 10, 10, 0, 1.0, 1.0, 0xffffff00)
  end
end

window = GameWindow.new
window.show

This example can be extended into a full Asteroids like like game where your ship has inertia. You should look at the source or an explanation on the gosu site.

There are all sorts of great resources out there including

Samples from the Gosu Showcase or

Falling Blocks or CptnRuby
Chingu an extension that seems to let us avoid re-writing common tasks
Chipmunk a physics library that makes it easy to do things like collision detection, gravity, etc

Even though we’re still writing Ruby, Gosu feels more like C++ Windows development I used to do long long time ago. I’m not sure if that’s inevitable and need to keep using Gosu to find out.

gamesalad

The last framework we’ve been working with is pretty different. GameSalad advertises it lets you “Create games for iPhone, iPad, Android, Mac, and HTML5. No coding required.”

It follows a model similar to what Adobe Flash uses where you have Scenes containing Actors. You write your programs in a visual editor by dragging and dropping Actors onto Scenes, Rules onto Actors and Behavior onto Rules. For instance if we have a starship actor and we drop these rules onto it

We will get our familiar spaceship that can move left and right

GameSalad is the least familiar to me but seems to be easiest for my kids to start working on. Not having to write any “code” or “do programming” makes it much easier to get started. It also can create iPhone or iPad games and I would never dream of exposing my kids to Objective-C.

What’s next?

We’ve started experimenting with all three of these tools and so far are having fun with all three. Hopefully we’ll figure out what works for us and perhaps try to write about it again in a few months.

After writing this I came across a recent NY Times article Programming for Children, Minus Cryptic Syntax and Scratch also sounds interesting so I may have to look into that sometime too.

Twitter is the new RSS Reader

2011-11-07T00:00:00-08:00

In 2008 I thought RSS was an awesome way to stay abreast of what’s going on, but now its 2011 and I find myself using Twitter more often than Google Reader to find new and interesting articles people have written. Readers tweet and retweet articles they find interesting which seems a lower barrier than leaving an “I like this” comment. As an author Twitter also gives you some idea of who is reading your posts and a way to connect with them.

Back in 2008 I created a blog aggregator site http://waywework.it to group the all the people I work with and promote others to share their thoughts. I was so excited I even wrote an article about it.

Now that its 2011, I’ve been asking myself how could I update http://waywework.it for the twitter world of today?

I decide that if we’re going to follow people on twitter that’s what my site should facilitate. When new posts come in it should tweet them letting you see them if you follow @WayWeWorkIT.

Enabling API access to twitter

Once I had this I added the twitter gem to my application. I have to give a shout out John Nunemaker for writing this fantastic gem which made my task so simple.

In the Gemfile

gem 'twitter'

I created a new twitter account @WayWeWorkIT and registered an application at https://dev.twitter.com/apps so I had my OAuth and access tokens.

The only trick was I had to go Application Settings tab and configure it for Read and Write access then regenerate the tokens.

Now that I had the keys and tokens from twitter I had to tell my application to use them without hardcoding them in my code. This took two steps. First, configuring the app to read the tokens from the environment in config/initializers/twitter.rb. Yes I am somewhat paranoid about accidentally tweeting from development but that if Rails.env.production? should save me.

if Rails.env.production?
  Twitter.configure do |config|
    config.consumer_key       = ENV['TWITTER_CONSUMER_KEY']
    config.consumer_secret    = ENV['TWITTER_CONSUMER_SECRET']
    config.oauth_token        = ENV['TWITTER_OAUTH_TOKEN']
    config.oauth_token_secret = ENV['TWITTER_OAUTH_TOKEN_SECRET']
  end
end

Secondly, setting the tokens on the heroku environment (I typed the real tokens instead of the XXXXXXXX’s).

$ heroku config:add TWITTER_CONSUMER_KEY=XXXXXXXX
$ heroku config:add TWITTER_CONSUMER_SECRET=XXXXXXXX
$ heroku config:add TWITTER_OAUTH_TOKEN=XXXXXXXX
$ heroku config:add TWITTER_OAUTH_TOKEN_SECRET=XXXXXXXX

We can test it out (after deploying with git push heroku)

$ heroku console
>> Twitter.tweet('http://waywework.it aggregates blog articles')
=> # some big object returned
>> Twitter.user_timeline('wayweworkit').first.text
=> "http://t.co/FCmUQdc3 aggregates blog articles"

Great we just tweeted our first tweet for the world to see.

Updating WayWeWork.IT to tweet new posts

The app periodically scans the rss feeds it tracks and when it sees a new post it creates it in the app’s database.

First we add a twitter_username to each feed we’re tracking

class AddTwitterUsernameToFeeds < ActiveRecord::Migration
  def change
    add_column :feeds, :twitter_username, :string
  end
end

Then, add an after_create callback to tweet each time we create a new post.

class Post < ActiveRecord::Base
  after_create :tweet

  delegate :twitter_username, :to => :feed

  def twitter_username_with_at_sign
    "@#{feed_twitter_username || 'WayWeWorkIT'}
  end

  # See https://dev.twitter.com/docs/tco-link-wrapper/faq#Will_t.co-wrapped_links_always_be_the_same_length
  # We should query instead of hardcoding 20
  def short_url_length
    20
  end

  def tweet
    if Rails.env.production?
      non_title_part_of_tweet = " #{'x'*short_url_length} via #{twitter_username_with_at_sign}"
      max_title_length = 140 - non_title_part_of_tweet.length

      tweet = "#{title.truncate(max_title_length)} #{url} via #{twitter_username_with_at_sign}"
      Twitter.update(tweet)
    end
  end
end

Again with the “if Rails.env.production?” paranoia? You do know that you can never be too paranoid :)

With the twitter gem its one line to tweet Twitter.update(tweet). The rest of it is to shorten the title so twitter’s 140 character limit wont cut off the url or the author’s name.

Once this is in we’ll start seeing tweets like

Go ahead and follow @WayWeWorkIT on twitter and you’ll start seeing these blog posts.

Using BDD and the email_spec gem to implement Email

2011-10-31T00:00:00-07:00

When implementing email functionality, the email_spec gem is something I’ve decided I can’t live without. It makes it so easy to write RSpec specs and Cucumber features around your email that you have no excuse not to. Today I’m going to go through an example how I recently used BDD to send an email in an app I was working on.

When I think about email and my Rails environments this is how I typically want them to behave.

test should not send emails and allow us to write specs or features against them
development should not send emails but provide a UI to view what would be sent on localhost
staging should not send emails but provide a UI to view what would be sent on a server
production should send real emails to real people

Last week I wrote about using letter opener to View Sent Email on a Server (without actually sending anything) describing how the letter_opener gem helps us in the development and staging environments. This article focuses on using email_spec gem in the test environment.

Our Sample Project

Let’s imagine we are working on a new startup in stealth mode. We want to generate buzz and prepare for a beta launch. We’re hiding the fact its all vaporware with a splashy homepage where people can request an invitation to the beta and records their email in our database. This is the same example as my article last week and there’s a live demo at http://awesome-site-staging.heroku.com/.

What we’ll do today is not just record the email address but also send a “thanks for your interest” email to the user. We’re going to do this in BDD fashion bouncing back and forth between Cucumber Scenarios and RSpec Unit Tests

Writing a cucumber scenario
While the scenario fails
1. Write a failing rspec spec
2. Write code to make it pass

The Cucumber scenario tells us what should be accomplished and when it fails it we use that to tell us what unit test we should write.

Adding the email_spec gem

We add it to our Gemfile

group :test do
  gem 'email_spec'
end

We bundle and use the email_spec generator to let it initialize itself.

$ bundle
$ rails g email_spec:steps

There’s also a manual step to get cucumber to load the email_spec gem. We need to create a file features/support/email_spec.rb

require 'email_spec/cucumber'

Our first Cucumber Scenario

We know that when a user requests an invitation they should get an email so we write that requirement as a Cucumber Scenario. features/request_an_invitation.feature

Feature: Build excitement for this vaporware
  In order to drum up interest
  As a user
  I will receive an exciting email when I request an invitation

  Scenario: Someone requests an invitation and receives an email
    Given I am on the home page
     When I request an invitation for "gullible@lemmings.com"
     Then "gullible@lemmings.com" should receive 1 email
      And they open the email
      And they should see the email delivered from "alex@awesome-startup.com"
      And they should see "Invitation request for Awesome New Startup received" in the email subject
      And they should see "Dear gullible@lemmings.com," in the email text part body
      And they should see "We have received your request " in the email text part body
      And they should see "Please check back at http://awesome-site-staging.heroku.com" in the email text part body

We run it and it tells us we have a couple of missing steps.

1 scenario (1 undefined)
9 steps (7 skipped, 2 undefined)
0m0.006s

You can implement step definitions for undefined steps with these snippets:

Given /^I am on the home page$/ do
  pending # express the regexp above with the code you wish you had
end

When /^I request an invitation for "([^"]*)"$/ do |arg1|
  pending # express the regexp above with the code you wish you had
end

Oh right, the training wheels came off in cucumber-rails v1.1.1 and we don’t have web_steps.rb anymore. Let’s write these steps using capybara in features/step_definitions/invite_steps.rb

Given /^I am on the home page$/ do
  visit root_path
end

When /^I request an invitation for "([^"]*)"$/ do |email|
  visit root_path
  fill_in 'email', :with => email
  click_button 'Request Invitation'
end

We run one more time and get a failure we expect. Its telling us we haven’t written any code to implement the scenario yet!

  Then "gullible@lemmings.com" should receive 1 email
  expected: 1
       got: 0 (using ==) (RSpec::Expectations::ExpectationNotMetError)
  ./features/step_definitions/email_steps.rb:52:in `/^(?:I|they|"([^"]*?)") should receive (an|no|\d+) emails?$/'
  features/request_an_invitation.feature:9:in `Then "gullible@lemmings.com" should receive 1 email'

Dropping into RSpec unit tests

Our failing feature tells us what we need to implement so we drop down to the unit test level and start implementing it with TDD. The feature tells us an email should be be generated so let’s go. We’ll add to our invites_controller_spec specifying that it should create and deliver an InviteMailer. spec/controllers/invites_controller_spec.rb

require 'spec_helper'

describe InvitesController do
  describe 'PUT #update' do
    let(:invite) { mock(:email => 'someone@someco.com', :save => true) }
    let(:invite_mailer) { mock }
    before do
      InviteMailer.should_receive(:invite_requested).with(invite).and_return(invite_mailer)
      invite_mailer.should_receive(:deliver)
      Invite.should_receive(:new).with('email' => 'someone@someco.com').and_return(invite)
      post :create, :invite => { :email => 'someone@someco.com' }
    end
    it { should redirect_to root_url }
    it { should set_the_flash.to("Thanks for your interest someone@someco.com.  You will hear from us soon.") }
  end
end

Of course we get an error uninitialized constant InviteMailer. We fix that by creating the mailer (it doesn’t do anything yet) app/mailers/invite_mailer.rb

class InviteMailer < ActionMailer::Base
end

The error changes

Failure/Error: InviteMailer.should_receive(:invite_requested).with(invite).and_return(invite_mailer)
  (<InviteMailer (class)>).invite_requested(#<RSpec::Mocks::Mock:0x82c96210 @name=nil>)
      expected: 1 time
      received: 0 times

We add the code to our controller to create and deliver the email in app/controllers/invites_controller.rb

class InvitesController < ApplicationController
  def create
    @invite = Invite.new(params[:invite])
    if @invite.save
      InviteMailer.invite_requested(@invite).deliver
      redirect_to root_path, :notice => "Thanks for your interest #{@invite.email}.  You will hear from us soon."
    else
      render :action => "new"
    end
  end
end

Are we done?

Checking the Cucumber Scenario…

The RSpec unit tests pass now but the Cucumber features are still failing.

When I request an invitation for "gullible@lemmings.com"            # features/step_definitions/invite_steps.rb:5
  undefined method `invite_requested' for InviteMailer:Class (NoMethodError)
  ./app/controllers/invites_controller.rb:10:in `create'
  (eval):2:in `send'
  (eval):2:in `click_button'
  ./features/step_definitions/invite_steps.rb:8:in `/^I request an invitation for "([^"]*)"$/'
  features/request_an_invitation.feature:8:in `When I request an invitation for "gullible@lemmings.com"'

Duh our InviteMailer doesn’t do anything.

Back down to the unit tests

We write our spec/mailers/invite_mailer_spec.rb. We need to include some EmailSpec modules so we have access to its matchers.

require 'spec_helper'

describe InviteMailer do
  include EmailSpec::Helpers
  include EmailSpec::Matchers

  describe '.invite_requested' do
    let(:invite) { Factory.build :invite, :email => 'someone@someco.com' }

    describe 'one email to one user' do
      subject { InviteMailer.invite_requested(invite) }
      it { should deliver_to     invite.email                                                  }
      it { should deliver_from   'alex@awesome-startup.com'                                    }
      it { should have_subject   "Invitation request for Awesome New Startup received"         }
      it { should have_body_text "Dear someone@someco.com,"                                    }
      it { should have_body_text "We have received your request"                               }
      it { should have_body_text "Please check back at http://awesome-site-staging.heroku.com" }
    end
  end
end

Of course it fails because we still haven’t implemented anything. Let’s add some code to app/mailers/invite_mailer.rb

class InviteMailer < ActionMailer::Base
  def invite_requested(invite)
    @invite = invite
    mail :to      => invite.email,
         :from    => 'alex@awesome-startup.com',
         :subject => 'Invitation request for Awesome New Startup received'
  end
end

and we can use haml to format the body in app/views/invite_mailer/invite_requested.text.haml

== Dear #{@invite.email},
We have received your request to be invited into our awesome site. We'll let you know as soon as its available.
Please check back at http://awesome-site-staging.heroku.com
You must be very excited!
Thanks
An Awesome New Startup

We run the rake one more time and …

We’re Done

Everything passes - the specs and the features!

$ rake
ruby -S rspec <a long list of _spec.rb files>
............

Finished in 2.64 seconds
12 examples, 0 failures

ruby -S bundle exec cucumber  --profile default
Using the default profile...
Feature: Build excitement for this vaporware
  In order to drum up interest
  As a user
  I will receive an exciting email

  Scenario: Someone requests an invitation and receives an email
    Given I am on the home page
    When I request an invitation for "gullible@lemmings.com"
    Then "gullible@lemmings.com" should receive 1 email
    And they open the email
    And they should see the email delivered from "alex@awesome-startup.com"
    And they should see "Invitation request for Awesome New Startup received" in the email subject
    And they should see "Dear gullible@lemmings.com," in the email body
    And they should see "We have received your request" in the email body
    And they should see "Please check back at http://awesome-site-staging.heroku.com" in the email body

1 scenario (1 passed)
9 steps (9 passed)
0m0.293s

I hope you’ll consider using the email_spec gem and BDD the next time you have to add email to your app.

Using Letter Opener to View Sent Email on a Server (without actually sending anything)

2011-10-24T00:00:00-07:00

When developing email functionality you don’t want to send real emails to real people before in production. At the same time you need to send them to ensure they are formatted correctly and contain the proper information. You can (and should) write integration tests to verify this but that helps developers gain confidence, what can we do to show non-technical stakeholders that it all works?

Today I’m going to show you how to use Ryan Bates’ letter_opener gem to let you preview your emails without actually sending them. Ryan of course does the always awesome RailsCasts.

Let’s think about the different Rails environments and how we want them to behave with email.

test should not send emails and allow us to write specs or features against them
development should not send emails but provide a UI to view what would be sent
staging should not send emails but provide a UI to view what would be sent
production should send real emails to real people

Looking at this, Rails works really well for test with ActionMailer’s :test delivery method that stores the emails in the ActionMailer::Base.deliveries array so you can then use email_spec to write your specs. Production is also covered as long as you give it your mail server configuration. That leaves development and staging which look identical but we’ll see that they are slightly different. I’ll spend the rest of this article talking about how letter_opener lets us do what we want in these environments.

An example

Let’s imagine we are working on a new startup in stealth mode. We want to generate buzz and prepare for a beta launch. We’re hiding the fact its all vaporware with a splashy homepage where people can request an invitation to the beta and it sends a “thanks for your interest” email. Lastly, we want to test it in development and staging.

Here’s a live demo at http://awesome-site-staging.heroku.com/ if you want to dive in and start clicking.

After you request an invitation it shows a page like this

Logfile testing (we can do better)

When you fill in your email and click the Request Invitation button, our controller uses a mailer to create and deliver the email.

class InvitesController < ApplicationController
  def create
    @invite = Invite.new(params[:invite])
    @invite.save
    InviteMailer.invite_requested(@invite).deliver
    redirect_to root_path, :notice => "Thanks for your interest #{@invite.email}.  You will hear from us soon."
  end
end

The only way to tell whether the email worked is to scroll through the development.log until you see something like this

Sent mail to alex@alexrothenberg.com (20ms)
Date: Fri, 21 Oct 2011 15:09:16 -0400
From: admin@newstartup.com
To: alex@alexrothenberg.com
Message-ID: <4ea1c35cc3d5b_6693830916bc43754@Alex-Rothenbergs-MacBook-Pro.local.mail>
Subject: Invite requested
...
We have received your request to be invited into our awesome site. We'll let you know as soon as its available.
Please check back at http://awesome-site.heroku.com
You must be very excited!
Thanks
An Awesome New Startup

Ok if you’re a developer and you enjoy reading log files but letter_opener lets us do better.

Using letter_opener in development

Letter_opener provides us with a UI so we can view the emails right in our browser. It’s super easy to add this gem and I’ll just copy the instructions from its README

Preview email in the browser instead of sending it. This means you do not need to set up email delivery in your development environment, and you no longer need to worry about accidentally sending a test email to someone else’s address

Rails Setup

First add the gem to your development environment and run the bundle command to install it.

gem "letter_opener", :group => :development

Then set the delivery method in config/environments/development.rb

config.action_mailer.delivery_method = :letter_opener

Now any email will pop up in your browser instead of being sent. The messages are stored in tmp/letter_opener.

Once we’ve done that what happens when we use the site to request an invitation? A new tab opens up with the email right there. Now as a user we can tell it’s correct and any non-technical people on the team can feel their confidence rise.

How does letter_opener actually work?

Rails goes through the standard flow to create the mail object and when it’s ready to deliver the message it calls letter_opener’s deliver! method because we registered letter_opener as the action_mailer.delivery_method. Letter_opener saves the email to your file system as an html file then uses launchy to open it in a browser using the file:// protocol.

There will be a couple of problems once we move onto a server which brings us to staging.

Using letter_opener on staging

If you’re like me you probably have a staging environment where you or your stakeholders can validate your app before releasing it to production and your end users. There are two aspects of this environment that wont work with letter_opener the way it did in development

We need to use http:// not file:// to preview the emails because the browser is not on the same file system where the emails are written
We may not be able to write to the file system. For example if we have deployed to heroku.

I had to make some changes to letter_opener to support this kind of server environment. The fork is available at https://github.com/alexrothenberg/letter_opener/tree/on_a_server and I’ll update the article if my pull requests are merged back in.

We need to make a few changes to our application.

1 - Update our Gemfile to use the fork from github

  gem 'letter_opener',  :git => "git://github.com/alexrothenberg/letter_opener.git", :branch => "on_a_server"

2 - Add a debugging UI link so users can get to the “preview emails” page in something like layouts/application.html.haml

  = link_to 'Preview Emails', letter_opener_letters_path if Rails.env.staging?

3 - If you cannot write to the filesystem let letter_opener know in your config/environments/staging.rb

  config.action_mailer.delivery_method = :letter_opener
  LetterOpener.cannot_write_to_file_system!

Now we can see it all in action.

First, we request an invite.

Then, we click the link “view the Emails that users would have received” link at the bottom and see an “inbox” of everything the app sent.

Fincally clicking one message lets us preview it just as we did in devellopment

We are able to run on heroku without writing to the file system by using the FakeFS gem which simulates the filesystem in memory. FakeFS is designed to be used for testing and one caveat to be aware of is that your old emails will disappear if heroku recycles your dyno due to inactivity.

I hope you think letter_opener is useful and give it a try the next time you need to send email. Remember here’s a live demo at http://awesome-site-staging.heroku.com/ of the site we’ve been talking about here.

Chaining ActiveRecord Scopes from Different Models

2011-10-17T00:00:00-07:00

You probably know you can chain ActiveRecord scopes together and it combines it all together into one sql statement, but did you know that you can also use scopes from associated models in your chain?

Let me show you with an example. Let’s say we have a discussion site where users can post comments with two simple models:

class User < ActiveRecord::Base
  has_many :comments
end

class Comment < ActiveRecord::Base
  belongs_to :user
end

We could display a list of all users with User.all or even display them alphabetically with User.order(:name) (yes I’d create a scope if I were doing this for real).

What if I wanted to display the users sorted so that the ones with the most recent comment was first?

User.includes(:comments).merge(Comment.order('comments.created_at desc'))

There’s a lot going on there, let’s break it down.

User.includes(:comments) - tells active record to query both the users and comments tables
Comment.order('comments.created_at desc') - sorts the results by the date of the comment (we need to specify the table and column name since created_at is also a column on the users table)
merge(Comment.XXX) - lets us use a scope from the Comment model even though we’re dealing with Users

When ActiveRecord and ActiveRelation take all this and convert it into a sql statement it will join the users and comments tables and order by the comments.created_at column. Here’s the sql I actually get in irb (boy am I glad I didn’t have to type that sql myself!).

> User.includes(:comments).merge(Comment.order('comments.created_at desc'))
  SQL (0.4ms)  SELECT "users"."id" AS t0_r0, "users"."name" AS t0_r1, "users"."created_at" AS t0_r2,
                      "users"."updated_at" AS t0_r3, "comments"."id" AS t1_r0, "comments"."user_id" AS t1_r1,
                      "comments"."body" AS t1_r2, "comments"."created_at" AS t1_r3, "comments"."updated_at" AS t1_r4
               FROM "users"
               LEFT OUTER JOIN "comments" ON "comments"."user_id" = "users"."id"
               ORDER BY comments.created_at desc

Adding Scopes to make it usable

It works to type all that but in a real application you’d add scopes to make it easier to work with. Let’s do that!

class User < ActiveRecord::Base
  has_many :comments
  scope :by_most_recent_comment, includes(:comments).merge(Comment.most_recent_first)
  scope :with_recent_comments, includes(:comments).merge(Comment.recently(1.month.ago))
end

class Comment < ActiveRecord::Base
  belongs_to :user
  scope :most_recent_first, order('comments.created_at desc')
  scope :recently, lambda { |date| where('comments.created_at >= ?', date) }
end

Now we can write some nice simple scopes like

User.by_most_recent_comment to get all users sorted so the ones with recent comments are at the top
User.with_recent_comments to get all users who have commented in the past month
User.with_recent_comments.by_most_recent_comment to get users who have commented in the past month sorted by the date of their most recent comment.

Happy scoping!

Ammeter: The Way to Write Specs for Your Rails Generators

2011-10-10T00:00:00-07:00

Generators got a complete makeover with Rails 3 making them much easier to write but they’ve been very hard to test if you’re using RSpec. That’s changed now with the Ammeter Gem which lets you write RSpec specs for your generators.

Who writes generators?

Unless you’ve writing a gem you probably haven’t created a generator, but I bet you’re using one someone else created. If you’ve ever typed

“rails g rspec:install” and a spec director appeared
“rails g cucumber:install” and gotten a features directory
“rails g model post title:string body:text” and gotten specs for your model
“rails g model post title:string body:text” and gotten mongoid models insead active record ones

There are a number of resources for writing generators using thor including the generators guide or railscast #218 and I’m not going to go into that here. If you’re using TestUnit, like the rails core team, you can use Generators::TestCase which is part of Rails - Devise has some good examples. For those of us using RSpec we can now use Ammeter.

Writing Specs with Ammeter

First you need to tell your gem to use ammeter by 1) adding it to our bundle and 2) making it accessible to our specs.

  # <YOUR_GEM_NAME>.gemspec
  s.add_development_dependency 'ammeter'

  # spec_helper.rb
  require 'ammeter/init'

Then we specify the behavior. We’ll look at an example using Mongoid’s config generator and its spec config_generator_spec. The generator’s usage is rails generate mongoid:config [DATABASE_NAME] [options].

# spec/generators/mongoid/config/config_generator_spec.rb
require 'spec_helper'

# Generators are not automatically loaded by Rails
require 'rails/generators/mongoid/config/config_generator'

class Rails::Application; end
class MyApp::Application < Rails::Application; end

describe Mongoid::Generators::ConfigGenerator do
  # Tell the generator where to put its output (what it thinks of as Rails.root)
  destination File.expand_path("../../../../../../tmp", __FILE__)
  before { prepare_destination }

  describe 'no arguments' do
    before { run_generator  }
    describe 'config/mongoid.yml' do
      subject { file('config/mongoid.yml') }
      it { should exist }
      it { should contain "database: my_app_development" }
    end
  end

  describe 'specifying database name' do
    before { run_generator %w(my_database) }
    describe 'config/mongoid.yml' do
      subject { file('config/mongoid.yml') }
      it { should exist }
      it { should contain "database: my_database_development" }
    end
  end
end

There’s some boilerplate setup you’ll need at the top of your spec:

Since this spec file is in spec/generators it automatically uses ammeter
Generators are not automatically loaded by Rails’ const_missing so we need to require it explicitly
destination tells the generator where to put its output (we add enough “../”s to get us out of the spec directory)
before { prepare_destination } clears the destination so each spec starts fresh (similar to active record rollback)

Now for the behavior:

run_generator runs the generator (optionally letting you pass in arguments)
file gives you access to a generated file
it should exist makes sure the file was generated
it should contain looks inside a generated file for a string or regex

When you’re generating migrations it is somewhat tricky because migration file names contain a timestamp. We need another example and will use acts_as_taggable-on’s migration generator We could write a spec for this as

# spec/generators/acts_as_taggable_on/migration/migration_generator_spec.rb
require 'spec_helper'

# Generators are not automatically loaded by Rails
require 'generators/acts_as_taggable_on/migration/migration_generator'

describe ActsAsTaggableOn::MigrationGenerator do
  # Tell the generator where to put its output (what it thinks of as Rails.root)
  destination File.expand_path("../../../../../tmp", __FILE__)

  before do
    prepare_destination
    Rails::Generators.options[:rails][:orm] = :active_record
  end
  describe 'no arguments' do
    before { run_generator  }

    describe 'db/migrate/acts_as_taggable_on_migration.rb' do
      subject { file('db/migrate/acts_as_taggable_on_migration.rb') }
      it { should be_a_migration }
    end
  end
end

You can see much of the same setup and then the new matcher

it { should be_a_migration } which adds a timestamp to the filename

Why ‘Ammeter’?

An Ammeter is a measuring instrument used to measure the electric current in a circuit. Generators produce electricity and your specs measure your generators … cute huh :)

Feedback Welcome

Try Ammeter for your generators, I hope you find it useful. If it doesn’t meet your needs fork away - Ammeter is on github. I welcome any feedback, issues or pull requests.

Upgrading An Old Rails Rails 2.1.1 to 3.1 on Heroku

2011-10-03T00:00:00-07:00

I recently had to upgrade an old app and it went pretty smoothly so I thought I’d share how I did it.

I thought about two approaches to this upgrade

I could slowly add modernity to the existing app by adding bundler, upgrading rails, upgrading rspec, etc.
I could create a new Rails 3.1 project and copy the code and specs into this new project.

I downloaded the old code and immediately remembered the pain of getting an app up and running without bundler. This convinced me that #1 would be harder than it needed to be so I decided to go with approach #2. It worked out pretty well so I’m going to share exactly what I did.

Create a new Rails 3.1 project

$ rails new waywework
      create
      create  README
      create  Rakefile
      # and a lot more...
         run  bundle install
Fetching source index for http://rubygems.org/
Using rake (0.9.2)
# and a lot more...
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.

Add gems from my app

Because the old app comes from the days before bundler I had to look in config/environment.rb for lines like config.gem 'atom' and guess what else I needed (I knew I was using rspec).

#Gemfile
gem 'atom'
group :development, :test do
  gem "rspec-rails"
  gem "factory_girl_rails"
  gem "haml-rails"
  gem 'faker'
end

Now we can make sure our empty app works (even though it does nothing)

$ cd waywework
$ rails g rspec:install
      create  .rspec
      create  spec
      create  spec/spec_helper.rb
$ rake db:migrate
$ rake
No examples matching ./spec/**/*_spec.rb could be found

Add our existing code and specs

$ cp -r ~/old_waywework/app/controllers/* app/controllers
$ cp -r ~/old_waywework/app/models/* app/models
$ cp -r ~/old_waywework/app/helpers/* app/helpers
$ cp -r ~/old_waywework/app/views/* app/views
$ cp -r ~/old_waywework/db/migrate db
$ mv spec/spec_helper.rb new_spec_helper.rb
$ cp -r ~/old_waywework/spec/* spec
$ mv new_spec_helper.rb spec/spec_helper.rb

Now when we try to run the specs it tells us we have pending migrations so we run them

$ rake
You have 5 pending migrations:
  20081022162743 CreateFeeds
  20081022162832 CreatePosts
  20081031025747 IndexPublishedInPosts
  20081031143453 IndexFeedAuthor
  20081103143931 PublishedAsDatetime
$ rake db:migrate
==  CreateFeeds: migrating ====================================================
-- create_table(:feeds)
   -> 0.0025s
==  CreateFeeds: migrated (0.0026s) ===========================================

==  CreatePosts: migrating ====================================================
-- create_table(:posts)
   -> 0.0021s
==  CreatePosts: migrated (0.0023s) ===========================================

==  IndexPublishedInPosts: migrating ==========================================
-- add_index(:posts, [:published, :feed_id])
   -> 0.0015s
==  IndexPublishedInPosts: migrated (0.0017s) =================================

==  IndexFeedAuthor: migrating ================================================
-- add_index(:feeds, :author)
   -> 0.0007s
==  IndexFeedAuthor: migrated (0.0008s) =======================================

==  PublishedAsDatetime: migrating ============================================
-- change_column(:posts, :published, :datetime)
   -> 0.0086s
-- add_column(:posts, :updated, :datetime)
   -> 0.0006s
==  PublishedAsDatetime: migrated (0.0094s) ===================================
$ rake
...LOTS OF ERRORS...
activerecord-3.1.0/lib/active_record/base.rb:1083:
    in `method_missing': undefined method `named_scope' for #<Class:0x103875b28> (NoMethodError)

Now we’re getting some errors and are ready to get to work.

Update our code

Of course it doesn’t just work and we need to make a number of changes. The guides include an upgrade process section and the rails_upgrade plugin can help identify problems. Here are the ones I found.

Changing `named_scope` to `scope`

With Rails 3 the syntax for scopes changed from named_scope to scope. So we search-and-replace in our models. Once we do this the specs run, but many are failing. A lot are failing because the routes.rb syntax changed in Rails 3.

RAILS_ROOT => Rails.root

The constant RAILS_ROOT no longer exists so we need to search-and-replace that with Rails.root.

Routes.rb

# OLD config/routes.rb
ActionController::Routing::Routes.draw do |map|
  map.posts_by_author 'author/:id', :controller=>'posts', :action=>'by_author'
  map.posts_by_month 'month/:year/:month', :controller=>'posts', :action=>'by_month'
  map.atom_feed '/atom', :controller => "posts", :action=>'index', :format=>'atom'

  map.resources :feeds
  map.root :controller => "posts"
end

We change it to:

# NEW config/routes.rb
WayWeWork::Application.routes.draw do
  match 'author/:id' =>'posts#by_author', :as => :posts_by_author
  match 'month/:year/:month' =>'posts#by_month', :as => :posts_by_month
  match '/atom' => 'posts#index', :as => :atom_feed, :format => :atom

  resources :feeds

  root :to => 'posts#index'
end

RSpec Routing Specs

We have a bunch of routing specs that are failing because RSpec changed the syntax around routing specs.

# OLD spec/controllers/feeds_routing_spec.rb
it "should map #index" do
  route_for(:controller => "feeds", :action => "index").should == "/feeds"
end
it "should generate params for #index" do
  params_from(:get, "/feeds").should == {:controller => "feeds", :action => "index"}
end

The routing spec must be in folder called routing

# NEW spec/feeds_routing_spec.rb
it "should generate params for #index" do
  get("/feeds").should route_to('feeds#index')
end

RSpec `stub!`

Another syntax change in RSpec from using stubs to stub!

# OLD
it 'should ...' do
  #...
  feed.stubs(:puts)
  feed.get_latest
end

We get an error undefined method 'stubs' for \#<Feed:0x105d572e8>

# NEW
it 'should ...' do
  #...
  feed.stub!(:puts)
  feed.get_latest
end

View Specs

Hmm I’m surprised I had written view specs. Nowadays I would write cucumber features and never write view specs. That being said there were a few things I needed to change to get the view specs passing, all related to RSpec syntax changes.

I was passing @ variables to the view with assigns[:feed] = @feed = stub_model(Feed) and today you need to do that with assign(:feed, @feed = stub_model(Feed). Also when you call render "/feeds/edit.html.erb" it used to work but now tries to render a partial. Plain old render will work as long as you’ve put the filename in your describe like describe "/feeds/edit.html.erb" do

Deploying to Heroku

The old app was deployed to a VPS using Capistrano. Today I prefer to use Heroku and there were a few changes I had to make to get it working there.

Add pg to my Gemfile to support Postgresql
Configure assets (since we cannot write to the filesystem) ** Added config.assets.compile = true in my production.rb ** Added therubyracer to my Gemfile
Delete my old Capfile

Now I was able to deploy to heroku and the new app is up and running. All this done in less than a day!

Talk Ruby to a Ruby Class instead of JSON to an HTTP Service

2011-09-26T00:00:00-07:00

Software as a service (SaaS) is a great thing. I love that other people are providing services and I don’t have to implement them myself. I can use Airbrake for error notifications and even Twitter for communication. It frees me to focus on what’s unique about my app. Its great that they all work with open standards like HTTP, JSON and XML. But what I like even more is not having to think about HTTP, JSON or XML! When writing a Ruby application I want to think about Ruby. The services I really like provide a gem that hides the transport api details and lets me write plain ruby.

The Airbrake gem lets me write Airbrake.notify(ex)
The Twitter gem lets me write Twitter.user_timeline("sferik")

If you are creating a service yourself or even using someone else’s service it’s not too hard to create your own client gem. Let’s look at an example how we can do that.

An Example: User Directory Service

Imagine you’re building a user directory that lets you list users, create users, update users, show users, delete users, you know, the standard RESTFUL actions. For instance we could create a user and list all users like this

[~]$ curl -F 'user[name]'='Mickey Mouse' http://user-directory.example.com/users.json
{"user":{"name":"Mickey Mouse","id":1001}}

[~]$ curl http://user-directory.example.com/users.json
[{"user":{"name":"Mickey Mouse","id":1001}}]

This is great to use from CURL but writing the code to talk http form fields and parse json will get pretty tiring. I’d much rather have a UserDirectory::User object that behaved similarly to an ActiveRecord model. When we’re done we want to be able to have it work this

user = UserDirectory::User.create(:name => 'Jenny', :phone => '867-5309')
# => #<UserDirectory::User:0x10239b0b8 @name="Jenny", @phone="867-5309", @id=1001>
user.name
# => 'Jenny'
user.id
# => 1001

UserDirectory::User.all
# => [ #<UserDirectory::User:0x102375638 @name="Jenny", @phone="867-5309", @id=1001>  ]

Building our client gem

We’ll want to wrap this client into a gem so it can be reused by many applications. I prefer to use bundler and its bundle gem command to start my gem. I’ll assume you know how to do that - check out this railscast if you’ve never done it before.

We’re going to get started by building a model class to represent the user without actually connecting it to our service yet.

module UserDirectory
  class User
    include ActiveModel::Validations
    validates_presence_of :name

    attr_accessor :name, :phone
    def initialize(attributes={})
      attributes.each do |name, value|
        send("#{name}=", value)
      end
    end
  end
end

I hope you didn’t think all model classes had to subclass ActiveRecord::Base! For me a model represents a concept in the application domain and does not have to map to a database table!

The magic in here is that we’re mixing in the ActiveModel::Validations module. This lets us add the same validations we would to a “regular” ActiveRecord model - in this case validates_presence_of :name. When Rails 3.0 came along much of the functionality of ActiveRecord was extracted into ActiveModel which lets you make any ruby object feel like ActiveRecord and as with most other topics Ryan Bates has put together a great railscast.

When we try it out it behaves as we expect.

# A valid user
user = UserDirectory::User.new(:name => 'alex')
# => #<UserDirectory::User:0x12b79fff0 @name="alex">
user.valid?
# => true

# an invalid user
user = UserDirectory::User.new(:phone => '555-1212')
# => #<UserDirectory::User:0x12b78ecf0 @phone="555-1212">
user.valid?
# => false
user.errors
# => #<ActiveModel::Errors:0x12b788f80 @base=#<UserDirectory::User:0x12b78ecf0
#      @errors=#<ActiveModel::Errors:0x12b788f80 ...>, @validation_context=nil, @phone="555-1212">,
#      @messages=#<OrderedHash {:name=>["can't be blank"]}>>

The next step is to make our model interact with the service using HTTP and JSON. I’m going to be using the HTTParty gem. It is much easier to use than net/http and automatically parses json or xml into a ruby hash for us.

First, we’ll add a create method that tells the service to create a new user as long as we pass our validations and then returns a User instance.

module UserDirectory
  class User
    # all the existing code and...

    def attributes
      { :name => name, :phone => phone }
    end

    include HTTParty
    base_uri 'http://user-directory.example.com'

    def self.create attributes
      user = new(attributes)
      return user unless user.valid?

      response = post('/users.json', :body => user.attributes)
      raise "#{response.code}: Better error handling please" unless response.success?
      self.new(response.parsed_response)
    end
  end
end

We can do the same for an all method that returns an array of all users stored in the service.

module UserDirectory
  class User
    # all the existing code and...

    def self.all
      response = get('/users.json')
      raise "#{response.code}: Better error handling please" unless response.success?
      response.parsed_response.map do |user_attributes|
        self.new user_attributes
      end
    end
  end
end

Let’s try it out.

# No users yet
UserDirectory::User.all
# => []

# Create a user
UserDirectory::User.create(:name => 'alex', :phone => '555-1212')
# => #<UserDirectory::User:0x12c52f0a8 @name="alex", @phone="555-1212">

# Now ask the service again
UserDirectory::User.all
# => [#<UserDirectory::User:0x12b8890d8 @name="alex", @phone="555-1212">]

This is great! We have a class that’s easy to use, behaves like a normal model but actually talks JSON and HTTP to a remote service. We could continue along the same lines to implement the other methods we need like find & destroy but I’m not going to bore you with that here. Instead I’ll switch gears and talk about building a fake service to eliminate the need to have the actual service running during development and testing.

Building a fake service

There are a few reasons we want to build a fake service.

It’ll be faster to not make any network calls
It’s a lot of overhead to start a local copy of the service during development
It’ll be easier to test various failures (500 errors and the like)
It allows us to setup and clear the data any way we want

How do we create a fake service? Luckily there’s a gem for that! The ShamRack gem intercepts http calls before they leave our app and redirects them to a local Rack App we’ll create. We’ll create a simple sinatra app that implements the UserDirectory Service API and embed it in our gem.

First things first, add the gem to our gem’s user_directory_client.gemspec

gem.add_dependency "sham_rack"

Now we can

require 'sinatra'
require 'sham_rack'

module UserDirectory
  class FakeService < Sinatra::Base

    ###################
    # ShamRack methods
    def self.activate!
      ShamRack.mount(self, "user-directory.example.com", 80)
    end
    def self.deactivate!
      ShamRack.unmount_all
    end


    ###################
    # Sinatra methods
    configure do
      set :raise_errors, true
      set :show_exceptions, false
    end

    USER_JSON = {:name=>'Jenny', :phone=>'867-5309'}.to_json

    get '/users.json' do
      content_type 'text/json'
      [USER_JSON] # :hardcoded list of 1 user
    end

    # :create new user
    post '/users.json' do
      content_type 'text/json'
      USER_JSON # pretend to create and return hardcoded user
    end
  end
end

Its very simple and will always return the same hardcoded user but that might be enough. We turn the fake service on and off with calls to UserDirectory::FakeService.activate! and UserDirectory::FakeService.deactivate!. Let’s take a look.

UserDirectory::FakeService.activate!
# => PeopleServices::FakePeopleService

UserDirectory::User.all
# => [#<UserDirectory::User:0x12c52f0a8 @name="jenny", @phone="867-5309">]

UserDirectory::User.create(:name => 'alex', :phone => '555-1212')
# => #<UserDirectory::User:0x12b8731e8 @name="jenny", @phone="867-5309">
UserDirectory::User.create(:name => 'pat')
# => #<UserDirectory::User:0x12b86cc58 @name="jenny", @phone="867-5309">

UserDirectory::User.all
# => [#<UserDirectory::User:0x12b865cf0 @name="jenny", @phone="867-5309">]

This is interesting. Its fast and eliminates the dependency, but its all hardcoded! When we created 2 users we still got the same “Jenny” user every time. The good news is the fake service is just a class we wrote so we can make is as complex as we need. Perhaps what we have here is enough for you and you’re all done but we’ll assume you want something a bit more realistic.

We’re going to create a quick array to simulate persisting our users.

require 'sinatra'
require 'sham_rack'

module UserDirectory
  class FakeService < Sinatra::Base

    # ShamRack methods ... remain unchanged

    # Sinatra methods ... changed to use our new "business logic"
    get '/users.json' do
      self.class.users.to_json
    end

    post '/users.json' do
      create_user(params)
    end

    ###################
    # Some "business logic"
    # the worlds simplest db :)
    def self.users
      @users ||= []
    end
    def create_user attributes
      attributes['id'] = rand(10000)
      self.class.users << attributes.dup
      attributes.to_json
    end
  end
end

One last time we’re going to try it out.

UserDirectory::FakeService.activate!
# => PeopleServices::FakePeopleService

# We start off empty
UserDirectory::User.all
# => []

# Create some users - the attributes correctly change
UserDirectory::User.create(:name => 'alex', :phone => '555-1212')
# => #<UserDirectory::User:0x12b85f300 @name="alex", @phone="555-1212">
UserDirectory::User.create(:name => 'pat')
# => #<UserDirectory::User:0x12b857880 @name="pat">

# Now we have our 2 users
UserDirectory::User.all
# => [#<UserDirectory::User:0x12b851318 @name="alex", @phone="555-1212">, #<UserDirectory::User:0x12b84adb0 @name="pat">]

This is now looking almost like a real service and will be very useful as we do our development. One last enhancement is that it’ll be nice to test what happens when the service has errors.

require 'sinatra'
require 'sham_rack'

module UserDirectory
  class FakeService < Sinatra::Base
    # Everything else unchanged...

    def self.fail_next_request!
      @fail_next_request = true
    end
    def self.should_fail_request?
      should_fail = @fail_next_request
      @fail_next_request = false
      should_fail
    end

    # Sinatra before filter
    before do
      halt 500, 'We were told to fail!' if self.class.should_fail_request?
    end
  end
end

One last time we’ll try it out.

UserDirectory::FakeService.activate!
# => true
UserDirectory::FakeService.fail_next_request!
# => true
 
UserDirectory::User.create :name => 'Alex'
RuntimeError: 500: We were told to fail!
	from /Users/alex/user_directory_client/lib/user_directory/user.rb:25:in `create'
	from (irb):56

Now go off and create a client for any service you create and be sure to include a fake service!

Running a Private GemServer inside the Firewall

2011-09-16T00:00:00-07:00

rubygems.org has made it so easy to publish a gem for the world to use but what do you do when your gem is proprietary and you only want to publish it within your company?

This is something I’ve just been through at my company and thought I’d share the steps I went through. We need to

Setup an inside-the-firewall gem server
Configure our gems to deploy to it
Configure our apps to use it

Setup an inside-the-firewall gem server

The first thing you have to decide is what gemserver to use. Rubygems.org has a helpful page called running your own gemserver that basically lists 3 choices in a goldilocks situation.

too small - gem server is a command built into rubygems

This works but you need to log onto the server to install a new gem and it serves all gems on the system not just your proprietary ones
too big - rubygems.org is open source so we could deploy it on our own server

This seems pretty complex to setup and even they tell you to “consider checking out Geminabox”
just right - gem in a box is a simple sinatra app to allow you to host your own in-house gems

This is easy to setup, has a web interface and supports a command line to remotely publish new gems.

geminabox is what I decided to go with.

The readme on github describes the server setup for geminabox and it just worked. The only thing to keep in mind is that you cannot use bundler as then you will only serve the gems in the bundle instead of the gems you publish. I spent some time adding bundler before realizing that was a bad idea and backing it out.

Once geminabox is up and running you can view your gems at your internal url and you’ll see the gem server homepage showing you it has no gems.

The easiest thing is to add a new gem by clicking “Upload Another Gem” and selecting a .gem file from your hard drive (I picked diagnostics-0.0.1.gem in the image below).

Once you click uppload you should see your gem on the page.

At this point we could start using this gem server in our apps but before we talk about that let’s automate the manual process we just went through to add a gem.

Configure our gems to deploy to the gem server

I’ve been using bundler to create my gems with the bundle gem command and one of the features that gives you is a set of nice rake tasks. Check out the New Gem with Bundler Railscast to learn how it works.

$ bundle gem my_awesome_gem
      create  my_awesome_gem/Gemfile
      create  my_awesome_gem/Rakefile
      create  my_awesome_gem/.gitignore
      create  my_awesome_gem/my_awesome_gem.gemspec
      create  my_awesome_gem/lib/my_awesome_gem.rb
      create  my_awesome_gem/lib/my_awesome_gem/version.rb
Initializating git repo in /Users/alex/my_awesome_gem

Let’s look at the tasks we’ve got.

$ cd my_awesome_gem
$ rake -T
rake build    # Build my_awesome_gem-0.0.1.gem into the pkg directory
rake install  # Build and install my_awesome_gem-0.0.1.gem into system gems
rake release  # Create tag v0.0.1 and build and push my_awesome_gem-0.0.1.gem to Rubygems

rake build and rake install do their work locally but rake release is what you call when you’re done and ready to release your gem into the wild. This task will push your changes to github, create a git tag, build your gem package and deploy it to http://rubygems.org. We need to do something to change that last part so it deploys to our private gem server instead of rubygems.org.

Let’s spend some time looking into bundler to figure out how rake release works. The magic all happens inside a file lib/bundler/gem_helper.rb

It defines a :release rake task
Which calls release_gem
Which calls rubygem_push
Finally this will call gem push pkg/my_awesome_gem-0.0.1.gem which pushes to http://rubygems.org. We’ve found the behavior we need to change.

geminabox adds a custom rubygems command called inabox so you can deploy a gem with the command gem inabox pkg/my-awesome-gem-1.0.gem. Unfortunately bundler does not seem to have a convenient way to change this so we’re going to monkey patch bundler Bundler::GemHelper#rugygem_push method to use the geminabox command instead. (please let me know if you have a better idea)

We’ll add our monkey patch to our Rakefile since its called by a rake command.

# Rakefile in your my_awesome_gem gem

# Monkey patch Bundler gem_helper so we release to our gem server instead of rubygems.org
module Bundler
  class GemHelper
    def rubygem_push(path)
      gem_server_url = 'http://gems.intranet.mycompany.com'
      sh("gem inabox '#{path}' --host #{gem_server_url}")
      Bundler.ui.confirm "Pushed #{name} #{version} to #{gem_server_url}"
    end
  end
end

You can see this will call gem inabox ... so we also need to add geminabox to our gem’s bundle. We do this in the .gemspec as a development dependency

# my_awesome_gem.gemspec in your gem

Gem::Specification.new do |s|
  .. lots of other stuff ...

  s.add_development_dependency "geminabox"
end

Now when we call rake release it will push the gem to our private server instead of the public one. Let’s see:

$ rake release
my_awesome_gem 0.0.1 built to pkg/my_awesome_gem-0.0.1.gem
Tagged v0.0.1
Pushed git commits and tags
Pushed my_awesome_gem 0.0.1 to http://gems.intranet.mycompany.com

Now when we go to the gem server site, we can see our new awesome gem in the list

The gem is there an you can use install it with a command like gem install my_awesome_gem --source http://gems.intranet.mycompany.com

Using your Gem Server from an application

We’ve just seen how we can use the source option to tell rubygems where to look when installing our gem by hand, but in a modern application we all use bundler and a Gemfile to manage our gems so how do we tell bundler to user our private gemserver for our private gems? Its super simple, you just need to add a source to the top of your Gemfile

source "http://gems.intranet.mycompany.com/"
source :rubygems

# regular old gems come from rubygems.org
gem "rails"
gem "rack"
gem "haml"

# my private gem comes from my private gemserver
gem 'diagnostics'

Now when we run bundle it looks in our private gem server as well as the public rubygems.org. Now that that you’ve got my_awesoem_gem you’re ready to add awesomeness to your app.

$ bundle
Fetching source index for http://gems.intranet.mycompany.com/
Fetching source index for http://rubygems.org/
Using rake (0.9.2)
Using activesupport (3.1.0)
Installing my_awesome_gem (0.0.1)
...etc..

How Bundler Groups relate to the Rails Environment

2011-07-18T00:00:00-07:00

Recently I’ve seen more and more Gemfiles that organize gems into groups and it got me wondering how bundler knows which groups to load. For the most part two things happen

At install time - Bundler includes a capistrano task that installs all gems except those only in the development or test groups on your server
At execution time - Rails tells bundler to load the default gems and those specific to your environment (development, staging or production)

How Bundler installs gems into your bundle

To tell bundler to use bundler on the server all you need to do is add the one line below to your Capfile

require 'bundler/capistrano'

This creates a capistrano task bundle:install that ultimately runs something like the command below on your server

bundle install --gemfile /srv/my_app/releases/20110715204318/Gemfile --path /srv/my_app/shared/bundle
               --deployment --quiet --without development test

Okay so it ran a bundle install but what really happened? Let’s take that command one piece at a time.

--gemfile /srv/my_app/releases/20110715204318/Gemfile tells it to use our Gemfile, that makes sense.
--path /srv/my_app/shared/bundle tells it where to put the bundle. Let’s see what that means.

It looks like it created all the rubygems directories for to isolate the gems for this project (very similarly to rvm gemsets)

    $ ls /srv/my_app/shared/bundle/
    ruby
    $ ls /srv/my_app/shared/bundle/ruby/
    1.8
    $ ls /srv/my_app/shared/bundle/ruby/1.8/
    bin  cache  doc  gems  specifications

--quiet hmm what else can I say
--without development test Aha so here’s where it tells bundler to skip the development and test groups. so allall gems outside a group or in a group other than development or test are installed.

How does Bundler remember these settings when it loads Rails and tries to load the bundle? It saves them away in a .bundle directory cat .bundle/config shows us

---
BUNDLE_FROZEN: "1"
BUNDLE_DISABLE_SHARED_GEMS: "1"
BUNDLE_WITHOUT: development:test
BUNDLE_PATH: /srv/my_app/shared/bundle

Now we understand how Bundler and Capistrano work together during a deployment to setup the bundle and install gems on the server. Let’s take a look at what happens when our app starts up.

How Rails and Bundler load your gems according to the Rails Environment

In your config/application.rb, right near the top, you have a line like this.

# If you have a Gemfile, require the gems listed there, including any gems
# you've limited to :test, :development, or :production.
Bundler.require(:default, Rails.env) if defined?(Bundler)

Rails tells bundler to require all the gems in the :default group and also the current Rails.env group. It uses the .bundle/config file to know where the gems are installed and find them. So that’s how the gems appropriate for your environment get automatically loaded when Rails starts.

What if you create a gem group that doesn’t correspond to any Rails env?

This is the problem that started me down this investigation. I came across a Gemfile with a group called cruise like this

group :cruise do
  gem 'metric_fu'
end

It was working meaning our cruise server ran metric_fu but why?

We weren’t using capistrano to run bundle install and instead just checked whether we were on our cruise server and ran the command bundle install in our Rakefile. Aside: We are looking into Jenkins as a continuous integration server that supports bundler This explains why the metric_fu was installed into our bundle (there was no --without so all gems are installed)
When our Rails app starts it would not load metric_fu becuase Rails.env will never be cruise when the application.rb line Bundler.require(:default, Rails.env) runs. We had worked around that by doing the require ourselves.

   require 'metric_fu'

While this does work in that our cruise build works it has the downside of installing metric_fu (and all the gems it depends on) on our production server! That’s because the bundler/capistrano task installs all gems not marked development or test and since metric_fu is marked cruise it gets installed. Now Rails will not load it so its not that bad but its still not good. We can take a quick look on our server to verify

$ ls /srv/my_app/shared/bundle/ruby/1.8/specifications/metric_fu-2.0.1.gemspec
shared/bundle/ruby/1.8/specifications/metric_fu-2.0.1.gemspec
$ ls /srv/my_app/shared/bundle/ruby/1.8/gems/metric_fu-2.0.1
HISTORY  lib  MIT-LICENSE  Rakefile  README  spec  tasks  TODO

Fortunately this is really simple to fix, we just need to change our Gemfile and move metric_fu into the test group

group :test do
  gem 'metric_fu'
end

My advice it do not create any gem groups that do not correspond to your Rails environments as that seems to be what the bundler-capistrano and bundler-rails integrations expect.

Backbone.js Makes Building JavaScript Applications Fun

2011-02-11T00:00:00-08:00

Like many developers I’ve had a long, complicated relationship with Javascript. Especially with libraries like jquery it’s incredibly easy to add interesting behavior to your pages, but unless you’re very careful its also likely that you’ll end up with a mess of spaghetti javascript. I know as I’ve gotten myself into that mess and abandoned many projects because they were just too hard to change. All this has changed with some of new libraries out there that help you write your javascript following the MVC pattern.

Today I’m going to talk about backbone.js and show how it helped me and two friends build a rich one-page application to understand an exception stack trace. Backbone.js describes itself as supplying structure to JavaScript-heavy applications by providing models with key-value binding and custom events, collections with a rich API of enumerable functions, views with declarative event handling, and connects it all to your existing application over a RESTful JSON interface.

There are a number of tutorials and examples available out there to get you started. The basic idea is to uses backbone.js to organize your code into models, views and controllers.

What is CodeBuddy

CodeBuddy is the application I’ll describe. It helps you navigate an exception stack raised by your Rails app or any other stack you paste in. I worked on it with Pat Shaughnessy and Daniel Higginbotham and Pat produced a super article describing what CodeBuddy is and how to use it.

There is some server-side code that replaces the Rails Show Exceptions page and syntax highlights a snippet of source code for each line in the stack but for this article I’m going to ignore that and focus on the interactive page and the javascript behind it.

Below is a picture showing a stack trace the background and the code snippet for the currently selected line in front. Now a picture is okay but to really get a sense of it I suggest you follow this link to experience Code Buddy in action - try pressing ↑ or ↓ or double clicking a few lines in the stack then press s.

There’s a lot going on here and if we had tried building this before discovering backbone.js we probably would have had created a mess - mixing javascript and html that quickly would have become hard to change. I’m going to show you how backbone let us separate the models from the views and create something that was not hard to grow.

Let’s get into the technical details!

Organizing our data in Backbone Models

We want to follow good OO design principles and thinking about this our exception stack is really just a few objects:

A Stack
- has many Addresses
- knows which Address is selected
Each Address has
- path to a file
- line number
- snippet of code

We can build this as a JSON object like

var stackJson = {
  "stack_frames": [
    { "path": "/Users/alex/ruby/github/test_app/app/controllers/users_controller.rb",
      "line": 5,
      "code": "class UsersController < ApplicationController\n..."
    },
    { "path": "/Users/alex/.rvm/.../lib/action_controller/metal/implicit_render.rb",
      "line": 4,
      "code": "module ActionController\n..."
    }
  ],
  "selected": 0
}

The first step is to turn this into backbone models.

// Stack, Address and Addresses
CodeBuddy.backbone.Stack = Backbone.Model.extend({
  initialize: function() {
    this.set({
      addresses: new CodeBuddy.backbone.Addresses(this.get('stack_frames')) 
    })
  }
})

CodeBuddy.backbone.Address = Backbone.Model.extend({
})

CodeBuddy.backbone.Addresses = Backbone.Collection.extend({
  model:CodeBuddy.backbone.Address
})

The way we build a model in backbone is by extending Backbone.Model. When extending we can add custom behavior if we want. In our example we tell the Stack to contain a collection of Address objects in the Addresses collection. We use the backbone collection framework to define Addresses and tell it that it is a collection of Address model objects. These models will use the default backbone behavior for the rest which includes read/write access to its properties get or set.

Now that they’re defined, we’re ready to interact with these models. For example below is what you’d see using the Chrome javascript console (I’m showing the output as comments for readability)

// Using our models in a console
CodeBuddy.stack = new CodeBuddy.backbone.Stack({
  "stack_frames": [
    { "path": "/Users/alex/ruby/github/test_app/app/controllers/users_controller.rb",
      "line": 5,
      "code": "class UsersController < ApplicationController\n..."
    },
    { "path": "/Users/alex/.rvm/.../lib/action_controller/metal/implicit_render.rb",
      "line": 4,
      "code": "module ActionController\n..."
    }
  ],
  "selected": 0
})
// inherits.child

CodeBuddy.stack.get('addresses')
// inherits.child

CodeBuddy.stack.get('addresses').first().get('path')
// "/Users/alex/ruby/github/test_app/app/controllers/users_controller.rb"

CodeBuddy.stack.get('addresses').first().get('code')
// "class UsersController < ApplicationController
// ..."

We now have an object hierarchy with default behavior which we can (and will) extend in a bit but first let’s build some views and get a page we can look at.

Building the UI with Backbone Views

We’re going to build

A StackView that contains many AddressViews
Each AddressView will display a single line from the stack.
Each view will be tied to one of our model objects

It will look something like this:

We can start with a StackView that’s tied to the Stack model. Creating a backbone view is very similar to how we created our models - we extend Backbone.View and can override behavior if we want.

CodeBuddy.backbone.StackView = Backbone.View.extend({

  el: $("#stack"),

  initialize: function() {
    this.model.get('addresses').each(this.addOneAddress);
  },
  
  addOneAddress: function(address, index) {
    var view = new CodeBuddy.backbone.AddressView({model: address});
    this.$("#stack").append(view.render().el);
  }
})

In this case we overrode the initializer function to create an AddressView for each address. It also uses jQuery to add the AddressView's html within the page’s #stack element.

To do the iteration we use another powerful javascript library called underscore.js. Underscore.js gives us a ruby-like collection methods letting us write .each(this.addOneAddress). This will iterate over all the addressses calling the addOneAddress function on each one. Underscore.js also gives us erb-like templating we’ll use in the AddressView…let’s take a look at that view.

CodeBuddy.backbone.AddressView = Backbone.View.extend({
  tagName:  "li",

  template: _.template("<span class='container'><%= path %>:<%= line%></span>"),

  initialize: function() {
  },

  render: function() {
    var html = this.template(this.model.toJSON())
    $(this.el).html(html);
    return this;
  }
})

You can see the template and how it does look like erb. It defines the html that will be displayed for each address and its able to access properties in the model like path and line. The template gets applied in the render function with the line this.template(this.model.toJSON()). Finally, tagName is used to wrap this html in an li tag.

Now we can put it all together to see on a page. We can start with a toplevel page that has an element with stack, includes our models and views and tells them to load.

<html>
<body>
  <ul id="stack"></ul>
  <script src="javascripts/code_buddy.js" type="text/javascript"></script>
  <script>
      CodeBuddy.setup({
        "stack_frames": [
          { "path": "/Users/alex/ruby/github/test_app/app/controllers/users_controller.rb",
            "line": 5,
            "code": "class UsersController < ApplicationController\n..."
          },
          { "path": "/Users/alex/.rvm/.../lib/action_controller/metal/implicit_render.rb",
            "line": 4,
            "code": "module ActionController\n..."
          }
        ],
        "selected": 0
      })
  </script>
</body>
</html>

After the page loads and the views render it changes the #stack div to have all this

<ul id="stack">
  <li>
    <span class="container">/Users/alex/ruby/github/test_app/app/controllers/users_controller.rb:5</span>
  </li>
  <li>
    <span class="container">/Users/alex/.rvm/.../lib/action_controller/metal/implicit_render.rb:4</span>
  </li>
</ul>

So far this is a lot of framework and structure for a simple page but now is when it gets interesting and backbone reveals its true power!

Showing code for the selected address

Let’s say we want to mark one address selected and show the code for that one.

We can start by adding an element to the html called code-viewer

<html>
<body>
  <ul id="stack"></ul>
  <div id="code-viewer">
    <div id="code"></div>
  </div>
  <!-- all the javascript omitted for clarity -->
</body>
</html>

Now we want to create a StackView to put the right code in there

CodeBuddy.backbone.CodeView = Backbone.View.extend({
  el:$("#code-viewer"),
  
  initialize: function() {
  },
  
  render: function() {
    this.$("#code").html(CodeBuddy.stack.selectedAddress().get('code'))
  }
})

In this view we used our models to find which address was selected CodeBuddy.stack.selectedAddress() so let’s add that method.

CodeBuddy.backbone.Stack = Backbone.Model.extend({
  initialize: function() {
    // same code as before
  },

  addresses: function() {
    return this.get('addresses')
  },

  selectedAddress: function() {
    var selected = this.get('selected')
    return this.addresses().at(selected)
  }
})

That’s it, now we have a code view on top of the stack view and our code is still clean and well organized. I’m not afraid to keep going and make our next change.

Changing the selected address

Oh, we just got a new requirement to be able to scroll up and down through the stack and see the code window change as we go.

We add some more functions to our Stack Model so we can change the selection and we also tell it to call the selectionChanged function whenever the selected property changes. Calling a function on a property change is something that’s given to us with backbone events. Getting the view to update itself is also easy we just tell it to with CodeBuddy.codeView.render()

CodeBuddy.backbone.Stack = Backbone.Model.extend({
  initialize: function() {
    this.bind('change:selected', this.selectionChanged);
    // same code as before
  },

  setSelection: function(newSelected) {
    if (newSelected >= 0 && newSelected < this.addresses().size()) {
      this.set({ selected: newSelected })
    }
  },
  
  selectPrevious: function() {
    this.setSelection(this.get('selected') - 1)
  },
  
  selectNext: function() {
    this.setSelection(this.get('selected') + 1)
  }, 

  selectionChanged: function(x) {
    this.addresses().at(x.previousAttributes().selected).view.render()
    this.addresses().at(x.changedAttributes().selected).view.render()
    CodeBuddy.codeView.render()
  },
  
  // all the existing functions remain  
})

The last thing we need to do is give some way for the user to change the selection. We decided to use jQuery Hotkeys to bind to the up and down arrows (↑ or ↓).

CodeBuddy.setStackKeyBindings = function(){
  $(document).bind('keydown', 'up',   CodeBuddy.stack.selectPrevious)
  $(document).bind('keydown', 'down', CodeBuddy.stack.selectNext)
}

Conclusion

I hope this example has shown a little of the power of using backbone.js when you need to write a complex javascript application.
If you want to see the actual application its all on Github in code_buddy.js

This was our first project working with backbone.js and I’m sure there’s much more we can learn but we’ve found that it lets us write modular code with many small methods which feels much closer to writing Ruby than my previous forays into javascript.

Backbone.js makes writing a javascript application fun!

Moved Blog to Jekyll and GitHub Pages

2011-01-27T00:00:00-08:00

I just moved my blog to Jekyll and Github Pages which I hope will help me get back into the rhythm of posting regularly.

Why did I switch?

There were several things I wanted to change

Syntax highlighting for my code — Jekyll uses pygments which supports about a bazillion languages including “Gherkin” (cucumber) and blogger had nothing.
Authoring tools I wanted — The blogger wysiwyg editor never really worked for me and I’d prefer to use TextMate and markdown or textile anyway.
New site design — Making the move gave me a chance to redo my design. The old one felt cluttered and dated and I hope this new one is not.
Geek street cred — I like having everything on github and besides all the cool kids are doing it :)

Jekyll is a gem that converts a structured directory of pages and posts written in markdown, textile or html into static html files that can be served by apache or nginx. Of course there’s an official definition that explains it in more detail.

With Github Pages I just have to create a git repository alexrothenberg.github.com and with each push github runs jekyll to generate a static site it serves up at http://alexrothenberg.github.com

How did I create this new site?

1. Create an empty site

There are lots of sites others have created with jekyll to peruse for design ideas. I spent some time looking through them and eventually picked one to use as a template. I copied one and pushed it to a new project on github called alexrothenberg.github.com.

2. Get my existing content

First I had to get my old content out of blogger into a Jekyll site. There are instructions for migrating from many different platforms (including from blogger to jekyll). I followed the instructions to “import with vilcans’ Jekyll rss_importer” and just had to change the rss url to http://www.alexrothenberg.com/feeds/posts/full?alt=rss&max-results=1000 since I had more than 25 posts.

I now had my posts in the _posts folder. I copied these posts into my alexrothenberg.github.com/_posts and had a version of my blog with all my old posts.

3. Syntax highlighting code

I next did some experimentation with the old posts to see how the code highlighting worked. It’s very natural and in-the-flow.

While writing my article I just add a block like

{ % highlight ruby %}
class User
  def name
    [first, last].join(' ') 
  end
end
{ % endhighlight %}

and get this fantastic output

class User
  def name
    [first, last].join(' ') 
  end
end

Well not quite… It generated html with all sorts of css classes but I couldn’t find css stylesheets out there with the Vibrant Ink theme I like.
Through some trial and error I created my own vibrant_ink.css

4. Disqus comments

Because Jekyll generates static pages it has no commenting engine so I have to integrate with an external service. I decided to go with disqus.
Adding it into my site was simple but getting my old comments in turned out to be much harder than it should have been.

First I went to disqus.com, signed up for an account and used their admin too to create a site for www.alexrothenberg.com. They give you a bit of javascript to add to your page. I added this to my _layouts/post.html and I was done.

It looked like it would be just as easy to import my comments from blogger as they have a big import from blogger button on their admin site under Tools->Import/Export. I ran it and I suddenly had all my old comments. But this is where things started to break down. Blogger ties the comments to blogger profiles instead of an email address so none of my comments had avatar pictures next to them.

What followed was a frustrating journey to try and edit the comments using their admin tool. The first thing I tried was using their admin tool but it doesn’t let you edit a commenter’s email or url so I couldn’t stick email addresses in when I knew them (for example I know my own email). My next approach was to export the comments in disqus, edit the xml file it gave me and re-upload. The problem here is that the exported xml uses a different format than the import accepts although they don’t tell you that when you try importing the file you exported it just tells you 0 items imported - how annoying?!?

After a lot of trial and error I wound up with all my comments in an xml file that disqus could import. In case you want to do something similar here are the steps I followed

Create a temp site in disqus (I called mine alexrothenberg1)
Import the comments from blogger using the Disqus import tool
Export those comments to an xml file - I saved mine in alexrothenberg.github.com/_disqus_import/exported_comments.xml
Run my script alexrothenberg.github.com/_disqus_import/exported2import to migrate the format to a Word Press XFR and change the timestamp format.
Hand edit
alexrothenberg.github.com/_disqus_import/import_comments.xml to update the emails and urls for each comment (luckily I didn’t have that many old comments)
Delete the temp discus site and create a new one
Import my XFR file to the new site
Jekyll and Blogger use different formats for post urls (jekyll includes year/month/date while blogger is year/month) so we need to run the disqus Migrate Threads->Upload a URL Map tool to upload a file like alexrothenberg.github.com/_disqus_import/url.map

5. Mapping my domain name

At this point I have a blog site up at http://alexrothenberg.github.com and its only a quick step to start serving it at http://www.alexrothenberg.com.

Create a CNAME containing www.alexrothenberg.com in my project
Update the dns entry in my domain to create a CNAME record aliasing www to alexrothenberg.github.com

Authoring and Publishing

Now that the site is up and running it was time to write my first article (this one).

jekyll --auto --server

Starts the jekyll server running at http://localhost:4000 and monitors the files so it regenerates each time I save an edit
newpost Moved Blog to Jekyll and GitHub Pages

Creates a new empty post using a script I stuck in my project _bin (thanks al3x for this gist)
Start writing content in _posts/2011-01-27-moved-blog-to-jekyll-and-github-pages.md

Each time I hit save I can preview in a few seconds at http://localhost:4000
git commit -am 'wrote the article'
git push origin master

In a few seconds github processes the site into static files and its published for all the world to see.

Setting up a new Mac for Rails Development

2011-01-04T00:00:00-08:00

I just got a new MacBook (whoo hoo!) because my old machine dies (boo hoo) and thought I'd write down the steps I'm taking to set it up. I imagine this is more for myself than anyone else but keep reading if you like.

Some Apps
1. Google Chrome for browsing
2. TextMate for editing
3. Adium for IM

Homebrew for package management
1. Open a terminal and type
```
ruby -e "$(curl -fsSL https://gist.github.com/raw/323731/install_homebrew.rb)
```
2. Download and Install XCode from Apple

git & github for development

Install GIT
```
brew install git
```
create ssh keys and add to github following these instructions

Configure git

git config --global user.name "Alex Rothenberg"
git config --global user.email "alex@alexrothenberg.com"
git config --global color.ui true

RVM
1. Install rvm
2. add bundler to ~/.rvm/gemsets/global.gems and install it
```
echo bundler >> ~/.rvm/gemsets/global.gems
gem install bundler
```
3. install Ruby 1.8.7 and make it default
```
rvm install 1.8.7
rvm --default 1.8.7
```

Databases
1. MySQL
```
brew install mysql
```
2. Oracle (this is complicated but works if you follow the directions exactly TO THE LETTER without missing a single step!) http://blog.rayapps.com/2009/09/14/how-to-install-oracle-database-10g-on-mac-os-x-snow-leopard/. Then I did have to run these extra commands
  1. Add oracle env settings to my user
```
#.bashrc
## ORACLE 
export ORACLE_HOME=/Users/oracle/oracle/product/10.2.0/db_1
export DYLD_LIBRARY_PATH=$ORACLE_HOME/lib
export ORACLE_SID=orcl
PATH=$PATH:$ORACLE_HOME/bin
```
  2. Enable me to access the oracle files
```
sudo dscl localhost -append /Local/Default/Groups/oinstall GroupMembership alex
```
  3. SQL Developer is an Oracle GUI

Test with a project

git clone git://github.com/alexrothenberg/legacy_data.git
cd legacy_data
export MYSQL
export ORACLE
bundle install
rake

Finally I'm following the trend and setting up a dotfiles project on github so I don't have to recreate them in the future.

Combining two related fields in a url to prevent tampering

2010-10-11T00:00:00-07:00

Usually when you select an item from a search results list you only need to pass the id in the url and that's enough to lookup the rest of the details when processing the response. Recently I had a case where I couldn't lookup the item from the id because it wasn't in my database. I'm going to describe how I passed several pieces of information in a single url and what I did to ensure users could not be manipulate them independently.

My application was searching and displaying a list of companies then keeping track of which one the user selects.

When someone clicks on a company name it would save the company's id and display a nice message "Thanks for selecting Google Inc." Since the companies came from a slow external service I didn't want to call again, I passed both the id and name in the url.

I built a couple of simple haml files to show my company list. Each row had a link to display the company name and passed both the name and id in the url.

# search_results.html.haml
%table
  %tr
    %th Company Name
    %th Location
  = render @companies


# _company.html.haml partial
%tr
  %td= link_to company.name, company_path(:id=>company.id, :name=>company.name)
  %td= company.location

The controller code was also pretty straightforward.

class CompaniesController
  def show
    company = Company.new(:id=>params[:id], :name => params[:name])
    current_user.update_selected_company = company
  end
end

This worked great and worked, but then we started looking at the urls in our browser and noticed they looked like http://my.site.com/companies/60902413&name=Google+Inc..

Hmm ... What would happen if someone changed the name in the url? We tried loading http://my.site.com/companies/60902413&name=Some+Silly+Name. Uh-oh, our database now stored the selected company Some Silly Name with id #60902413. This could be confusing or worse a security risk where a clever (or malicious) user could store inaccurate information in our database.

Our solution was to combine the two fields into a single query parameter that was resistant to user tampering and is passed as a single unit. Fortunately Rails passes the session back and forth in a cookie with just that requirement. The session is a hash of many different key-value pairs that need to be encoded as a single cookie, it also contains sensitive information that should be resistant to tampering (and unreadable) and, most importantly it turns out to be something we could reuse.

Starting from the outside in what we wanted to do was rewrite our _company.html.haml partial view to put the single encrypted form of the company on the url.

%tr
  %td= link_to company.name, company_path(:id=>company.to_encrypted_s)
  %td= company.location

and parse that in the controller

class CompaniesController
  def action
    company = Company.from_encrypted_s(params[:id])
    current_user.update_selected_company = company
  end
end

Ok, but what do those to_encrypted_s and from_encrypted_s methods do?

class Company
  def to_encrypted_s
    Encryption.new.encrypt(:company_name=>company_name, :duns=>duns)
  end

  def self.from_encrypted_s encrypted_data
    Company.new Encryption.new.decrypt(encrypted_data)
  end
end

I still haven't told you how they work, we're just working down from the outside-in figuring out what other classes we'll need. So what does the Encryption class look like?

class Encryption
  def initialize
    secret = (Rails.version > '3.0') ?
             Rails.application.config.cookie_secret : 
             ActionController::Base.session_options[:secret]
    @verifier = ActiveSupport::MessageVerifier.new(secret, 'SHA1')
  end

  def encrypt message
    @verifier.generate message
  end

  def decrypt encrypted
    @verifier.verify encrypted
  end
end

We make use of the ActiveSupport::MessageVerifier class which as the documentation says "makes it easy to generate and verify messages which are signed to prevent tampering".

Now when we select a company we get a url with a crazy long id like http://my.site.com/company/BAh7BzoJZHVuc2kDE%2BAUOhFjb21wYW55X25hbWUiMEludGVybmF0aW9uYWwgQnVzaW5lc3MgTWFjaGluZXMgQ29ycG9yYXRpb24%3D--9027b2c449c0b4a1aea375cb1722fa9db8e56066. If someone were to try and change the really long id in the url the application will throw an ActiveSupport::MessageVerifier::InvalidSignature exception instead of saving bad data.

We've given up readability in the url but in exchange we're guaranteed the id and name we get in the controller go with each other.

Purging old cruise builds

2010-09-29T00:00:00-07:00

Continuous Integration is such an important idea and I love CruiseControl.rb as its so easy to use. However one thing that drives me crazy is that it keeps all builds forever. Over the past few years we run out of disk space every few months causing all our builds to start failing and some tedious manual work to delete some directories.

Today I decided to do something about it and wrote a script to remove all but the most recent builds. After all why do I want an old build lying around as that's what Git of Subversion is for!

If you want to give it a try you can download the script from http://github.com/alexrothenberg/purge_old_cruise_builds

This seems to be working for me and I'd love to hear if I'm making some assumptions you need to change to adapt it to your uses.

Use dashes not underscores in your domain name

2010-09-24T00:00:00-07:00

I recently ran into an issue where users hitting my development server with Internet Explorer kept getting redirected back to my login page no matter how many times they signed in. Everything worked fine for all other browsers I tested and it even worked fine on Internet Explorer when pointing at http://localhost.

After much investigating it turns out the problem lies in the name of my development server. I picked a name like http://myapp_dev.alexrothenberg.com and it was the underscore in myapp_dev that caused the problem. It turns out that IE will not set a session cookie for a domain with a hostname that contains an underscore.

According to microsoft this is not a bug but a feature

Q5: IE won’t set a cookie when the hostname/domain contains an underscore?

A: Correct. Technically, underscore is not a DNS character, and while Windows will let you use an underscore when naming your machine, it warns you that doing so may cause problems. One such problem is that WinINET blocks attempts to set cookies on such domains.

OK, so it looks like IE will let me go to the page but doesn't set the session cookie so my site sees each request as part of a new session by an unauthenticated user and redirects to the login page over and over.

I did a little more research and according to Wikipedia Microsoft is right and underscores are not allowed in hostnames

The Internet standards (Request for Comments) for protocols mandate that component hostname labels may contain only the ASCII letters 'a' through 'z' (in a case-insensitive manner), the digits '0' through '9', and the hyphen ('-') ...
While a hostname may not contain other characters, such as the underscore character (_), other DNS names may contain the underscore.

I'm not sure why IE allows the browsing but not the cookie as that turned this into a subtle bug that was especially hard to track down since Firefox, Safari and Chrome all set the cookie so my site works. From now on I'll have to remember not to use underscores but to use dashes instead like http://myapp-dev.alexrothenberg.com which should work across all browsers.

Its not too late to solve a Y2K bug - I found one in 2010

2010-03-14T00:00:00-08:00

I thought a historical opportunity had passed me by, but a full decade after the turn of the millennium in 2010 I found my first Y2K bug! If you, like me, were working in eCommerce (who remembers that buzzword?) in the late 1990s and thought you missed your chance I'm here to tell you its still not too late. As long as you work for the right company with the enough legacy technology around you can still find a Y2K bug of your own!

What was going on?

I was writing a new Rails application on top of an old Oracle database. The database had a constraint


graduationdate BETWEEN TO_DATE('01-JAN-1900','DD-MON-YYYY') AND TO_DATE('31-DEC-2100','DD-MON-YYYY')

To me this looked like a basic check to make sure the date was within a 300 year range as a basic sanity-check. I implemented an ActiveRecord validation in my model

validate :validate_graduation_date
def validate_graduation_date
  unless graduationdate.nil?
    earliest_date, latest_date = Date.parse('01-JAN-1900'), Date.parse('31-DEC-2100')
    errors.add(:graduationdate, 'must be between 01-JAN-1900 and 31-DEC-2100') unless graduationdate > earliest_date and graduationdate < latest_date  
  end
end

Of course I had unit tests that all passed

describe 'graduation date' do
  it 'should not allow July 4, 1776' do
    education = Factory.build :education, :graduationdate=>Date.parse('04-JUL-1776')
    education.should_not be_valid
  end
  it 'should allow Aug 1 1970' do
    education = Factory.build :education, :graduationdate=>Date.parse('01-AUG-1970')
    education.should be_valid
  end
  it 'should allow Mar 15 2000' do
    education = Factory.build :education, :graduationdate=>Date.parse('15-MAR-2000')
    education.should be_valid
  end
  it 'should not allow Jan 1, 2525' do
    education = Factory.build :education, :graduationdate=>Date.parse('01-JAN-2525')
    education.should_not be_valid
  end
end

BUT once I went into production with my legacy database I started getting errors. So I took a look at one of the rows giving me an error and discovered the graduation date was 01-JAN-00. AHA a 2-digit year. That doesn't look good and there were over 100,000 rows with this problem.

select count(*) from education where graduationdate = to_date('01-jan-1900')
COUNT(*)               
---------------------- 
116232

Once I realized that the database contained data for January 1st 1900 and I discovered the Oracle BETWEEN function is inclusive I was able to solve my problem by changing my ActiveRecord validation to have a >= and lt;=.

validate :validate_graduation_date
def validate_graduation_date
  unless graduationdate.nil?
    earliest_date, latest_date = Date.parse('01-JAN-1900'), Date.parse('31-DEC-2100')
    errors.add(:graduationdate, 'must be between 01-JAN-1900 and 31-DEC-2100') unless graduationdate >= earliest_date and graduationdate <= latest_date  
  end
end

I added some specs to test my boundary conditions

it 'should not allow Dec 31 1899' do
  education = Factory.build :education, :graduationdate=>Date.parse('31-DEC-1899')
  education.should_not be_valid
end
it 'should allow Jan 1 1900' do
  education = Factory.build :education, :graduationdate=>Date.parse('01-JAN-1900')
  education.should be_valid
end
it 'should allow Dec 31 2099' do
  education = Factory.build :education, :graduationdate=>Date.parse('31-DEC-2099')
  education.should be_valid
end
it 'should allow Jan 1 2100' do
  education = Factory.build :education, :graduationdate=>Date.parse('01-JAN-2100')
  education.should_not be_valid
end

So what's the moral?

I'm not sure but when you're have to integrate with a legacy system you should account for the absurd - in my case a job application from a candidate who graduated more than 100 years ago!

And if you feel like you missed out on a once-in-a-lifetime opportunity to work on Y2K bugs don't despair. They are still out there!

What to do when ActiveRecord thinks an Oracle key is a decimal

2010-01-27T00:00:00-08:00

I recently created a model for an existing database table using the legacy_data gem and was confused when my primary key showed up in scientific notation. It turned out the issue was due to sloppiness in the table definition and could be easily fixed once I understood what ActiveRecord was doing.

I created a Person model connected to the people table

class Person < ActiveRecord::Base
end

but when I went into script/console the primary key showed up as a BigDecimal when I expected an integer.

$ script/console 
Loading development environment (Rails 2.3.4)
>> Person.first.id
=> #

This wasn't what I wanted and would cause problems in my app when it tried to build a url with that id like http://localhost:3000/people/10024844425.0. The rails routing engine would see the .0, treat it as a format (like .xml or .json) and get confused. Let's look at why this is happening.

>> Person.columns_hash['id']
=> #<ActiveRecord::ConnectionAdapters::OracleEnhancedColumn:0x37391c0 @default=nil, @type=:decimal,
   #   @null=true, @name="id", @table_name="people", @scale=nil, @sql_type="NUMBER", @precision=nil,
   #   @primary=true, @forced_column_type=nil, @limit=nil>
>> Person.columns_hash['id'].type
=> :decimal
>> Person.columns_hash['id'].sql_type
=> "NUMBER"

We see that ActiveRecord is treating this column as a :decimal because it's sql_type is NUMBER. It turns out this is correct because an Oracle number is a decimal unless you specify it to have 0 digits after the decimal point (scale of 0). Here's the documentation from Oracle (the last sentence is my bold)

NUMBER Datatype

The NUMBER datatype stores fixed and floating-point numbers. Numbers of virtually any magnitude can be stored and are
guaranteed portable among different systems operating Oracle, up to 38 digits of precision.

The following numbers can be stored in a NUMBER column:
  * Positive numbers in the range 1 x 10-130 to 9.99...9 x 10125 with up to 38 significant digits
  * Negative numbers from -1 x 10-130 to 9.99...99 x 10125 with up to 38 significant digits
  * Zero
  * Positive and negative infinity (generated only by importing from an Oracle Version 5 database)

For numeric columns, you can specify the column as:
  column_name NUMBER 

Optionally, you can also specify a precision (total number of digits) and scale (number of digits to the right of the decimal point):
  column_name NUMBER (precision, scale) 

If a precision is not specified, the column stores values as given. If no scale is specified, the scale is zero.

Oracle guarantees portability of numbers with a precision equal to or less than 38 digits. You can specify a scale and no precision:
  column_name NUMBER (*, scale) 

In this case, the precision is 38, and the specified scale is maintained.

When you specify numeric fields, it is a good idea to specify the precision and scale. This provides extra integrity checking on input.

Let's look in my database and sure enough the ID is a number

$ sqlplus myusername/mypassword@localhost:1521/mydatabase.world
SQL*Plus: Release 10.2.0.4.0 - Production on Wed Jan 27 09:15:09 2010
Copyright (c) 1982, 2007, Oracle.  All Rights Reserved.

Connected to:
Oracle Database 10g Release 10.2.0.4.0 - Production
SQL> desc people;
Name        Null?    Type
----------------------------------------- -------- ----------------------------
ID                   NUMBER
NAME                 VARCHAR2(10)

If you are allowed to change your database you can create a migration like


$ script/generate migration change_person_id_to_integer
STUBBING MckinseyLDAP
      exists  db/migrate
      create  db/migrate/20100127145747_change_person_id_to_integer.rb

now edit the migration

class ChangePersonIdToInteger < ActiveRecord::Migration
  def self.up
    change_column(:people, :id, :integer)  
  end
  def self.down
    change_column(:people, :id, :decimal)  
  end
end

In my case there were other applications using this table and I was not allowed to change it so I implemented a fix in Ruby to tell my model to treat this column as an integer even though it was defined as a decimal in the database.

#config/initializers/legacy_data_type_coercion.rb
module LegacyDataTypeCoercion
  def set_integer_columns *col_names
    col_names.each do |col_name|
      columns_hash[col_name.to_s].instance_eval do
        @type = :integer
      end
    end
  end
end
ActiveRecord::Base.extend(LegacyDataTypeCoercion)


#app/models/person.rb
class Person < ActiveRecord::Base
  set_integer_columns :id
end

We defined a method set_integer_columns that will force ActiveRecord to treat the columns we specify as integers. In our Person model we declare :id is an integer column. Let's test it out!

$ script/console 
Loading development environment (Rails 2.3.4)
>> Person.first.id
=> 10024844425
>> Person.columns_hash['id'].type
=> :integer

Just as expected id is now an integer and we can go ahead building the rest of our application.

This is not an issue with all Oracle tables as if the column was defined as NUMBER(10) (with a precision and implicit scale of 0) then ActiveRecord will interpret it as an integer automatically based on the parentheses in the data type - i.e NUMBER(10) ActiveRecord or Oracle Enhanced).

Freezing a gem that has native extensions

2010-01-13T00:00:00-08:00

I like to freeze all the gems I use as we run in a shared hosting environment and need to our apps isolated from each other. Deployments are also handled by an operational team that does not intimately understand our applications so keeping our deployments to a single capistrano command cap deploy:migrations has been a big win for us. Freezing most gems is pretty straightforward and has been built in since Rails 2.1. When dealing with a gem that requires native extensions to be built there's only one additional step to add to your Capfil.

Let's say we want to localize hpricot which does include native C extensions.

First tell Rails about your gem by adding a config.gem line to your environment.rb


Rails::Initializer.run do |config|
  ...
  config.gem 'hpricot'
  ...
end

Now we can ask rails about its configured gems


  $ rake gems
  (in /Users/alexrothenberg/ruby/my_project)
   - [I] hpricot 

   I = Installed
   F = Frozen
   R = Framework (loaded before rails starts)

The 'I' means its hpricot is installed on my system but not frozen in the application. If you see '[]' instead you need to run sudo gem install hpricot (add '--source http://gemcutter.org' if necessary). At this point you could write some code to use hpricot and your application will work. But if hpricot (or the version you're expecting) is not installed on your production server you'll be in trouble.

To freeze the gem into your vendor directory run rake gems:unpack (optionally you can add 'GEM=hpricot' if you just want to unpack one gem).


  $ rake gems:unpack 
  (in /Users/alexrothenberg/ruby/my_project)
  Unpacked gem: '/Users/alexrothenberg/ruby/my_project/vendor/gems/hpricot-0.8.2'

We can ask rails again...


  $ rake gems
  (in /Users/alexrothenberg/ruby/my_project)
  The following gems have native components that need to be built
    hpricot  

  You're running:
    ruby 1.8.6.287 at /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby
    rubygems 1.3.2 at /Users/alexrothenberg/.gem/ruby/1.8, /Library/Ruby/Gems/1.8, /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/gems/1.8

  Run `rake gems:build` to build the unbuilt gems.

Oops our vendored gem is missing hasn't built the native extensions. Not to worry the message tells us what to do and we run rake gems:build


  $ rake gems:build
  (in /Users/alexrothenberg/ruby/my_project)
  Built gem: '/Users/alexrothenberg/ruby/mars-admin/vendor/gems/hpricot-0.8.2'
  alex-rothenbergs:mars-admin alexrothenberg$ rake gems
  (in /Users/alexrothenberg/ruby/my_project)
   - [F] hpricot 

  I = Installed
  F = Frozen
  R = Framework (loaded before rails starts)

We can ask rails again to see that the gem is now frozen and also look in our vendor folder


  $ rake gems
  (in /Users/alexrothenberg/ruby/my_project)
   - [F] hpricot 

  I = Installed
  F = Frozen
  R = Framework (loaded before rails starts)
  
  $ ls vendor/gems/hpricot-0.8.2/
  total 72
  -rw-r--r--   1 alexrothenberg  staff  4672 Jan 13 12:33 CHANGELOG
  -rw-r--r--   1 alexrothenberg  staff  1048 Jan 13 12:33 COPYING
  -rw-r--r--   1 alexrothenberg  staff  9216 Jan 13 12:33 README
  -rw-r--r--   1 alexrothenberg  staff  8242 Jan 13 12:33 Rakefile
  drwxr-xr-x   4 alexrothenberg  staff   136 Jan 13 12:33 ext/
  drwxr-xr-x   3 alexrothenberg  staff   102 Jan 13 12:33 extras/
  drwxr-xr-x   6 alexrothenberg  staff   204 Jan 13 12:39 lib/
  drwxr-xr-x  11 alexrothenberg  staff   374 Jan 13 12:33 test/

Everything looks good and you can check this into git and now have a frozen version of the hpricot gem stored with your application.
But if we stop here, when we deploy to our production server we'd be using the native extensions we built on your laptop which may not work on the server if you have one is 32bit and the other 64bit or you have different OS libraries installed or any number of other reasons.

To be safe, we need to rebuild the native extensions on the server when we deploy. This is not as hard as it sounds as rails gave us the rake task rake gems:build. We can ask capistrano to run that command on the server by adding the following to your Capfile.


after "deploy:finalize_update" do
  # build the native extensions for hpricot gem
  run "cd #{release_path} && #{rake} RAILS_ENV=#{rails_env} gems:build GEM=hpricot"
end

Now when capistrano deploys in with all the other messages you'll see something like


  ...
  * executing "cd /opt/apps/my_project/releases/20100108185109 && rake RAILS_ENV=production gems:build"
     servers: ["your.server.com"]
     [your.server.com] executing command
  ** [out :: your.server.com] (in /opt/apps/my_project/releases/20100108185109)
  ** [out :: your.server.com] Built gem: '/opt/apps/my_project/releases/20100108185109/vendor/gems/hpricot-0.8.2'
     command finished
  ...

So rails give us a few simple patterns to follow to freeze our gems in the vendor folder and with a few lines in you Capfile you can use this pattern to vendor a gem with native extensions.

Generate Models from Tables - Legacy Data Gem

2009-11-05T00:00:00-08:00

Today I'd like to announce the release of a gem I've been working on Legacy Data

Getting started on a Rails project with a large existing database can be daunting. How to you extract all the information that's
encoded in the database? Do you have to understand the entire data model before you get started? The models_from_tables generator
in the legacy_data gem can help! This generator looks into your existing database and generates ActiveRecord models based on the
information encoded in it.

How to use it

To generate an ActiveRecord model for each table in the database just type
script/generate models_from_tables
If you don't want all tables in the database tell it which table to model
script/generate models_from_tables --table-name comments
This uses any foreign_key constraints in the database to spider the database and model the comments table and all associated tables.
If you really only want the comments table tell it not to follow any foreign_keys
script/generate models_from_tables --table-name comments --skip-associated
If you use factory girl (and everyone should) it will generate a simple factory for each model it generates
script/generate models_from_tables --table-name comments --with-factories

(You do need to install the plugin gem install legacy_data as long as http://gemcutter.org is one of your gem sources)

Examples

Several examples come with the gem source in the examples folder on github. These include

A simple blog database tested with MySQL and Sqlite3
The Drupal 6.14 database tested with MySQL
The J2EE Petstore example tested with MySQL, Sqlite3 and Oracle

What kind of information can it extract from the database?

Associations

If the database contains foreign_key constraints it uses them to build has_many or belongs_to associations
in your ActiveRecord models

Validation constraints

It will generate the following types of validation constraints in your models

validates_uniqueness_of - For columns where the database has an index that enforces uniqueness
validates_presence_of - When the database column is non-nullable
validates_inclusion_of - For non-nullable boolean columns and custom constraints with a SQL rule flag IN ('Y', 'N')
validates_numericality_of - For integer columns (nullable and non-nullable)
custom validation - For custom SQL validation rules in the database it puts a placeholder in your model with the original SQL for you to translate into Ruby

Non-Rails naming conventions

Since the database is existing it's likely that it doesn't follow Rails naming conventions. Not to worry as the generator will
put the non-standard name into the generated models if it needs to.

What kinds of non-standard names can it generate?

Let's look at a sample output

class Post < ActiveRecord::Base

  set_table_name  :tbpost
  set_primary_key :postid
  
  # Relationships
  has_many :comments, :foreign_key => :postid

  # Constraints
  validates_presence_of :title, :body
  
end

Class Names - It named the model Post instead of the Rails convention Tbpost. The generator could not do this itself but knowing the conventions will often not apply to legacy databases it pauses after spidering the database giving you a chance to override the table to class name mapping. It generates a yaml file app/models/table_mappings.yml where you can verify or change any class name before proceeding to generate the models.
Table Names - It overrode the table name since the actual name tbpost does not match the Rails naming convention posts
Primary Keys - It overrode the primary key since the actual column postid does not match the Rails naming convention id
Foreign Keys - It overrode the foreign key on the comment table to be postid instead of the Rails naming convention id

PL/SQL debugging information now a part of Oracle Enhanced Adapter v1.2.2

2009-10-02T00:00:00-07:00

Raimonds Simanovskis has just published a version 1.2.2 of the Oracle Enhanced Adapter that includes the ability to capture dbms_output debug statements from your pl/sql code in the Rails log file. This is a bit of code that I wrote and blogged about a few months ago so not only do I think its useful but am very excited to have contributed to something many others use.

Using the Whenever gem to manage scheduled cron jobs without installing it on the server

2009-09-30T00:00:00-07:00

I've been using Javan's Whenever gem to manage scheduled jobs in my project and its fantastic!! There are many existing resources where you can learn more (readme, railscast google group) but I'd like to describe the specific way I'm using it

When the gem is not installed on my server
How administrators can use Capistrano to both schedule and unschedule your jobs
How to use a library such as the Oracle client that requires certain environment variables

At the end of the day we want to have 2 capistrano tasks we can run to have cron call a rake task of ours on the schedule we want.

  cap schedule_jobs
  cap unschedule_jobs

If we want to get fancy and pass a custom configuration argument that will be passed into the rake task.

  cap schedule_jobs SOME_CONFIGURATION=false

When the gem is not installed on my server

Let's start with the Capfile

#Capfile
desc "Schedule the jobs"
task :schedule_job, :roles => :app, :only => { :primary => true } do
  some_configuration = ENV['SOME_CONFIGURATION'] || true #default to true
  arguments = ["RAILS_ENV=#{rails_env}",
               "APP_PATH=#{current_path}",
               "SOME_CONFIGURATION=#{some_configuration}"].join(' ')
  run "cd #{current_path} && #{rake} whenever:update_crontab #{arguments}"
end

desc "Unschedule the jobs"
task :unschedule_job, :roles => :app, :only => { :primary => true } do
  run "cd #{current_path} && #{rake} whenever:update_crontab  UNSCHEDULE=true"
end

How does this differ from the example on the whenever site? Since the gem is not installed on the server we cannot call whenever from the command line so invoke the whenever:update_crontab rake task instead and to allow administrators to easily disable the scheduled jobs we define the unschedule_jobs capistrano task. Let's take a look at the whenever:update_crontab Rake task that gets this all done.

#lib/tasks/whenever.rake
namespace :whenever do
  desc "updates crontab with our scheduled jobs"  
  task :update_crontab => :load_whenever_gem do     
    Whenever::CommandLine.execute({:update=>true, :identifier=>'YOUR_APP_NAME'})
  end

  task :load_whenever_gem do     
    begin
      gem_dir_root = "#{RAILS_ROOT}/vendor/gems/"
      chronic_gem_dir = Dir["#{RAILS_ROOT}/vendor/gems/*"].detect do |subdir|
        subdir.gsub(gem_dir_root,"") =~ /^(\w+-)?chronic-(\d+)/ && File.exist?("#{subdir}/lib/chronic.rb")
      end
      require "#{chronic_gem_dir}/lib/chronic"

      whenever_gem_dir = Dir["#{RAILS_ROOT}/vendor/gems/*"].detect do |subdir|
        subdir.gsub(gem_dir_root,"") =~ /^(\w+-)?whenever-(\d+)/ && File.exist?("#{subdir}/lib/whenever.rb")
      end
      require "#{whenever_gem_dir}/lib/whenever"

    rescue MissingSourceFile => e
      raise "Cannot find Whenever or Chronic : #{e}"
    end
  end
end

The actual whenever:update_crontab task just does the same the command line does but unless you add config.gem 'whenever' to your environment (which I don't since whenever is not needed by my app at runtime) we also have the other task that loads whenever and chronic from the vendor/gems directory.

At this point we've gotten Capistrano calling a rake task to invoke whenever even though the gem is localized in my application but not installed on the server.

How administrators can use capistrano to both schedule and unschedule your jobs

The whenever gem does not have any support for unscheduling but it will schedule whatever is included in your schedule.rb file so if that file tells it to schedule nothing that's the same as unscheduling. Its easy to do that by wrapping the entire file with unless ENV['UNSCHEDULE'] (remember the UNSCHEDULE parameter we passed in the Capfile?)

#config/schedule.rb
unless ENV['UNSCHEDULE']

  module MyApp
    module Job
      class CronRakeTask < Whenever::Job::Default
        def output
          path_required
          "cd #{@path} && /usr/bin/env /usr/local/bin/cron_rake #{task} SOME_CONFIGURATION=#{ENV['SOME_CONFIGURATION']} RAILS_ENV=#{@environment}"
        end      
      end
    end
  end

  set :path,        ENV['APP_PATH'] || RAILS_ROOT 
  set :environment, RAILS_ENV || 'production'
  every 1.day, :at => '10:00pm' do
    command 'do_something', :class       => MyApp::Job::CronRakeTask, 
                            :environment => @environment, 
                            :path        => @path
  end
end

What's going on with the weird CronRakeTask? This brings us to the final point

How to use a library such as the Oracle client that requires certain environment variables

Cron loads its environment settings differently than an interactive shell and typically does not have all the environment variables you may have in your profile file. You could solve this by adding them to the top of your crontab file but I prefer to leave that file as simple as possible and create a wrapper script to call instead of rake. Basically the CronRakeTask does the same as Whenever's built in RakeTask except it calls /usr/local/bin/cron_rake instead of rake.

The cron_rake file just sets the environment variables I need then calls rake.

#!/bin/bash

export PATH=/usr/local/lib/ruby-enterprise/bin:$PATH
export ORACLE_HOME=/opt/oracle
export LD_LIBRARY_PATH=/opt/oracle:$LD_LIBRARY_PATH

rake $@

As I said in the beginning since I've been using the Whenever gem I no longer need to manually edit my crontab files ever and I can enable my jobs as part of my normal deployment process. Its wonderful and I think everyone should use it!

How to capture Oracle's dbms_output in your Rails log file

2009-08-19T00:00:00-07:00

I have been writing a rails application on top of a large existing Oracle database where each table has 5+ triggers that each call several stored procedures and each of those PL/SQL stored procedures is hundreds of lines long. Often a simple update statement fails with an ORA-xxxx exception coming from deep in the PL/SQL code and it can be tough to figure out what's gone wrong.

The usual way Oracle database folks figure out what's going on is to put print statements in their code. In oracle this looks like

  dbms_output.put_line('hi i hit this line of pl/sql');

When you're using an Oracle editor like TOAD or SQLDeveloper you have to turn output on and then will see anything that's printed.

  set serveroutput on;

This is great if you divide the application between Rails
and Database developers and assume each group can work independently to write perfect code but what about the real world!
Today I want to show you how I monkey patched the Oracle Enhanced Adapter to stick the dbms_output into
the rails log file.

Let's start with an simple example of a simple PL/SQL function that tells you if a string is more than 5 characters long (with some simple debugging print statements).

  CREATE OR REPLACE FUNCTION
  MORE_THAN_FIVE_CHARACTERS_LONG (some_text VARCHAR2) RETURN INTEGER
  AS
  longer_than_five INTEGER;
  BEGIN
    dbms_output.put_line('before the if -' || some_text || '-');
    IF length(some_text) > 5 THEN
      dbms_output.put_line('it is longer than 5');
      longer_than_five := 1;
    ELSE
      dbms_output.put_line('it is 5 or shorter');
      longer_than_five := 0;
    END IF;
    dbms_output.put_line('about to return: ' || longer_than_five);
    RETURN longer_than_five;
  END;

Now we can run the following in TOAD

  set serveroutput on;
  select MORE_THAN_FIVE_CHARACTERS_LONG('a long string') from dual;
  select MORE_THAN_FIVE_CHARACTERS_LONG('short') from dual;

And we get this output

  MORE_THAN_FIVE_CHARACTERS_LONG('ALONGSTRING') 
  --------------------------------------------- 
  1                                             

  1 rows selected

  before the if -a long string-
  it is longer than 5
  about to return: 1

  MORE_THAN_FIVE_CHARACTERS_LONG('SHORT') 
  --------------------------------------- 
  0                                       

  1 rows selected

  before the if -short-
  it is 5 or shorter
  about to return: 0

To get the same in our Rails app we just need to monkey patch the OracleEnhancedAdapter by copying what's below into your project as config/initializers/oracle_enhanced_adapter.rb

  module ActiveRecord
    module ConnectionAdapters
      class OracleEnhancedAdapter < AbstractAdapter
        DBMS_OUTPUT_BUFFER_SIZE = 10000  #can be 1-1000000
        DBMS_LINE_MAX_SIZE      = 1000  
        def enable_dbms_output
          @enable_dbms_output = true
          execute "BEGIN dbms_output.enable(#{DBMS_OUTPUT_BUFFER_SIZE}); END;"
        end
        def disable_dbms_output
          @enable_dbms_output = false
          execute "BEGIN dbms_output.disable(); END;"
        end
        def dbms_output_enabled?
          @enable_dbms_output
        end

        protected
          def log(sql, name)
            super(sql, name)
          ensure
            log_all_dbms_output if dbms_output_enabled?
          end

        private  
          def log_next_line_of_dbms_output
            dbms_output_text, status = @connection.exec "BEGIN dbms_output.get_line(:return, :status); END;", ' '*DBMS_LINE_MAX_SIZE, 1
            got_text = (status == 0)
            @logger.debug "DBMS_OUTPUT: #{dbms_output_text}" if got_text
            got_text
          end
        
          def log_all_dbms_output
            while log_next_line_of_dbms_output do
            end
          end
      end
    end
  end

To use it let's show a quick script/console session

  >> ActiveRecord::Base.connection.enable_dbms_output
  => []
  >> ActiveRecord::Base.connection.select_all("select MORE_THAN_FIVE_CHARACTERS_LONG('a long string') from dual")
  => [{"more_than_five_characters_long('alongstring')"=>1}]

And what's in log/development.log


    SQL (27.0ms)   BEGIN dbms_output.enable(10000); END;
    SQL (25.9ms)   select MORE_THAN_FIVE_CHARACTERS_LONG('a long string') from dual
  DBMS_OUTPUT: before the if -a long string-
  DBMS_OUTPUT: it is longer than 5
  DBMS_OUTPUT: about to return: 1

This is admittedly a very simple example but I have used this in a real application where I am updating several related ActiveRecord objects and seeing the DBMS_OUTPUT inline with the various SQL update statements has been extremely useful in tracking down a real bug in the PL/SQL procedure that has been in our system for over a year!

I've submitted this as a patch to the Oracle Enhanced Adapter so perhaps it will be included at some point so you wont have to do the monkey patching yourself. I was somewhat surprised not to find anything similar out there so if you know of something please leave a comment.

One year of blogging

2009-07-22T00:00:00-07:00

I just noticed that its been a year since my first post and thought I'd take a moment to reflect on the experience....

I set out with a goal of writing 2-3 posts a month and have managed to write 28 articles over 12 months so I feel pretty good about that. Most of the time I've gotten excited about something I accomplished or learned during the week. The process of writing it up turned out to be as much of a learning experience for myself as the original discovery. As they say "Teaching is the best way to learn"!

I've been most surprised how easy it is to get to the top of a Google Search results list (for very specific searches of course). With my Google Analytics tracking I can see more than 1/2 of my hits coming from search engines and not all of them are me searching for my own name :) Its been great when I've gotten comments from people I don't know (especially when the comments are positive).

Writing this blog has also helped turn my focus outward so I'm not only working on internal projects at my company but feel (in a small way) a part of the broader community.

I hope to continue in the coming year as I'm having a lot of fun so far ...

Using Github Through Draconian Proxies (Windows And Unix)

2009-06-25T00:00:00-07:00

I came across this great set of instructions on how to tunnel through a proxy to use github - http://returnbooleantrue.blogspot.com/2009/06/using-github-through-draconian-proxies.html

The proxy where I work is not quite so draconian that I need to follow these steps but its nice to have these instructions just in case :)

Metric_fu 1.1.0 released with a patch from me

2009-06-24T00:00:00-07:00

Metric_fu just released version 1.1.0 of their gem which I'm pleased to say includes a patch submitted by me. This is exciting as its the first time I've had my code included by someone I don't know in one of the open source projects I admire.

I was not originally going to make the patch as I thought I'd just hack around to fix the problem locally just enough to get it working but my friend Matt encouraged me to fix the root cause which turned out to be not too hard and got me into the metric_fu source. I always learn something when I read source from others I admire.

The process of submitting the patch was pretty easy and I plan to make it a habit when I run into issues with other gems. I,

Experienced a problem and decided not to live with it
Created a fork of metric_fu on github
Cloned locally and iteratively made my changes until I had my fix complete
Rebased my changes into a single commit (I wrote an article on this a few weeks ago)
Pushed my patch back to github on a branch
Submitted my patch back to Jake's repository as an issue

Solving the problem of gem dependencies with github names

I am using relevance-rcov instead of the rubyforge version of rcov as it has been maintained more recently and specifically fixed a segmentation fault problem I previously faced. The problem is that I couldn't install metric_fu as it put a gem dependency on 'rcov' in its gemspec. What seems broken is that there's no way to put a dependency on 'rcov' but have rubygems realize that 'relevance-rcov' is a fork of rcov so should satisfy the dependency.

I've seen a lot of talk recently about rubygems (including Why Using require ‘rubygems’ Is Wrong, RubyGems: Problems and (proposed) Solutions and Rip: A Next Generation Ruby Packaging System - Watch Out RubyGems!) but so far I haven't found a solution to the forking and naming problem emerge.

For Metric_fu I decided to change the install-time dependency to a runtime one. This solves my basic problem of not finding relevance-rcov but also allows for more flexibile use of metric_fu. If someone does not want to generate reek or rcov metrics why should they be forced to install those gems in order to use metric_fu? By defering the dependency until runtime we will never hit the dependency for those metrics we are not using.

You can look through the commit to see exactly how I did this.

Deleted the install time dependencies from metric_fu.gemspec
Added the runtime dependencies when a specific type of metric is instantiated by

Inserting a verify_dependencies! strategy step to the initialization in lib/base/generator.rb
Implementing verify_dependencies! in each of the subclasses for different metrics in lib/generators/*

Did it with TDD writing tests around everything before implementing my changes

Should gems use install time dependencies? Should a gem author decide which fork of a gem is required? What do you think?

Changing history with git rebase: How to combine several commits into one

2009-06-10T00:00:00-07:00

When I look at commits on github I'm always impressed at how concise they are. When I read a commit I can understand the intent of the change without getting distracted by the author's journey to get there. In contrast when I look at my commits they tend to be smaller and more incremental and meandering as I work my way down some false starts until I get to the solution I want. I'm guessing that I'm not alone in the way I work and recently discovered git rebase and an helpful tutorial showing how I can continue to work in my meandering style but package my changes to hide the journey before publishing to the world on github.

Let me show you what I mean with some changes I recently made to metric_fu. Over the course of a few days I made 6 small commits as you can see below.


> git log

commit d4b18b16e982ac57741f7a0a12cb085bb9b0e840
Author: Alex Rothenberg 
Date:   Mon Jun 1 09:55:37 2009 -0400

    reverted rakefile

commit ebda1cb67a2f0f0a85e51469d70919fa7c27d318
Author: Alex Rothenberg 
Date:   Mon Jun 1 09:49:05 2009 -0400

    refactoring of verify_dependencies

commit 3d45903b64a772fd09ec07bf69880bdb29ae4944
Author: Alex Rothenberg 
Date:   Fri May 29 20:40:49 2009 -0400

    runtime dependency check for all gems

commit 36e269bf8f0edccfb39cc767182406cdaa16a559
Author: Alex Rothenberg 
Date:   Fri May 29 20:24:16 2009 -0400

    logic for checking dependencies in generator base

commit bb2ee9a983c6adf54a8c95ee5851c6f8d3bffaba
Author: Alex Rothenberg 
Date:   Fri May 29 19:38:21 2009 -0400

    made rcov dependency figure itself out when generating rcov metrics

commit 750b000e5563e917f21eb1b7837e8001fb53f688
Author: Alex Rothenberg 
Date:   Fri May 29 16:42:12 2009 -0400

    removed gem dependency on rcov - to allow use of relevance-rcov or other github forks

commit d6af5089adce9eeed4916a155c3bdaeb4be6771a
Author: Randy Souza 
Date:   Sat May 16 08:47:37 2009 +0800

    Added a simple fix for cases where Saikuro results with nested information
    cause metrics:all to crash
    
    Signed-off-by: Jake Scruggs 

...

I am going to combine all of these into a single commit using the incredible power of git rebase. I find the last commit I do not want to change (the one made by Randy Souza on May 16th) and issue a git rebase command with that id.


> git rebase --interactive d6af5089adce9eeed4916a155c3bdaeb4be6771a

Now my editor comes up showing the 6 changes since then giving me options of what to do. Its very powerful, I can reorder commits, combine commits, eliminate commits. I'm going back in time to change the past!


pick 750b000 removed gem dependency on rcov - to allow use of relevance-rcov or other github forks
pick bb2ee9a made rcov dependency figure itself out when generating rcov metrics
pick 36e269b logic for checking dependencies in generator base
pick 3d45903 runtime dependency check for all gems
pick ebda1cb refactoring of verify_dependencies
pick d4b18b1 reverted rakefile

# Rebase d6af508..d4b18b1 onto d6af508
#
# Commands:
#  p, pick = use commit
#  e, edit = use commit, but stop for amending
#  s, squash = use commit, but meld into previous commit
#
# If you remove a line here THAT COMMIT WILL BE LOST.
# However, if you remove everything, the rebase will be aborted.
#

In my case I want to combine these into a single commit so I change all but the first pick command to a squash and save.


pick 750b000 removed gem dependency on rcov - to allow use of relevance-rcov or other github forks
squash bb2ee9a made rcov dependency figure itself out when generating rcov metrics
squash 36e269b logic for checking dependencies in generator base
squash 3d45903 runtime dependency check for all gems
squash ebda1cb refactoring of verify_dependencies
squash d4b18b1 reverted rakefile

# Rebase d6af508..d4b18b1 onto d6af508
#
# Commands:
#  p, pick = use commit
#  e, edit = use commit, but stop for amending
#  s, squash = use commit, but meld into previous commit
#
# If you remove a line here THAT COMMIT WILL BE LOST.
# However, if you remove everything, the rebase will be aborted.
#

It now gives me a chance to edit the new commit message which it defaults to the original messages concatenated together. This really is a single commit so if one of the original commits changed a file and a subsequent one undid the change that file will no longer appear in the list of modified files.



# This is a combination of 6 commits.
# The first commit's message is:
removed gem dependency on rcov - to allow use of relevance-rcov or other github forks

# This is the 2nd commit message:

made rcov dependency figure itself out when generating rcov metrics

# This is the 3rd commit message:

logic for checking dependencies in generator base

# This is the 4th commit message:

runtime dependency check for all gems

# This is the 5th commit message:

refactoring of verify_dependencies

# This is the 6th commit message:

reverted rakefile

# Please enter the commit message for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
# Explicit paths specified without -i nor -o; assuming --only paths...
# Not currently on any branch.
# Changes to be committed:
#   (use "git reset HEAD ..." to unstage)
#
#       modified:   Rakefile
#       modified:   lib/base/generator.rb
#       modified:   lib/generators/flay.rb
#       modified:   lib/generators/flog.rb
#       modified:   lib/generators/rcov.rb
#       modified:   lib/generators/reek.rb
#       modified:   lib/generators/roodi.rb
#       modified:   lib/generators/saikuro.rb
#       modified:   metric_fu.gemspec
#       modified:   spec/base/generator_spec.rb
#       modified:   spec/generators/flay_spec.rb
#       modified:   spec/generators/flog_spec.rb
#       modified:   spec/generators/reek_spec.rb
#

Let's edit this commit message to read something like


Changed gem dependencies from install-time in gemspec to runtime when each
of the generators is loaded.  This allows use of github gems (i.e.
relevance-rcov instead of rcov) and also allows you to install only the
gems for the metrics you plan on using.

                                     
# Please enter the commit message for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
# Explicit paths specified without -i nor -o; assuming --only paths...
# Not currently on any branch.
# Changes to be committed:
#   (use "git reset HEAD ..." to unstage)
#
#       modified:   Rakefile
#       modified:   lib/base/generator.rb
#       modified:   lib/generators/flay.rb
#       modified:   lib/generators/flog.rb
#       modified:   lib/generators/rcov.rb
#       modified:   lib/generators/reek.rb
#       modified:   lib/generators/roodi.rb
#       modified:   lib/generators/saikuro.rb
#       modified:   metric_fu.gemspec
#       modified:   spec/base/generator_spec.rb
#       modified:   spec/generators/flay_spec.rb
#       modified:   spec/generators/flog_spec.rb
#       modified:   spec/generators/reek_spec.rb
#

And that's it! Now when we check our history we see.


> git log

commit 6f389364eb972871867d3a71677b8eb7046541a2
Author: Alex Rothenberg 
Date:   Fri May 29 16:42:12 2009 -0400

    Changed gem dependencies from install-time in gemspec to runtime when each
    of the generators is loaded.  This allows use of github gems (i.e.
    relevance-rcov instead of rcov) and also allows you to install only the
    gems for the metrics you plan on using.

commit d6af5089adce9eeed4916a155c3bdaeb4be6771a
Author: Randy Souza 
Date:   Sat May 16 08:47:37 2009 +0800

    Added a simple fix for cases where Saikuro results with nested information
    cause metrics:all to crash
    
    Signed-off-by: Jake Scruggs 

...

And this commit is how I pushed this patch to github for all the world to see.

How to use dates in Rails when your database stores a string

2009-05-21T00:00:00-07:00

When working with Rails there's a lot of magic that happens behind the scenes to make it easy to do complex things. Most of the time you don't need to know how that "magic" works but there are times when things don't work as expected and its helpful to dig in and understand what Rails is doing under the covers so you can change how it works. Did I just say "change how Rails works"?!? I did! Rails is opinionated software that seeks to lead you down the golden path but there are legitimate times when you have to veer off that path and Rails lets you do so. I find this most often happens to me when I'm dealing with an existing legacy database which is not setup as Rails would like.

Today I'm going to go through an example that happened to me recently when I had an existing database that stored some dates in a text column but I needed to treat them as dates in my UI. I couldn't change the type of that column as there was another legacy application that expected it to be text.

Using Dates the Rails Way

First let's look at how easy it is to work with dates when you can follow the Rails Way. Let's create a new project and add a scaffolded Person object with a date attribute called birthday.

rails date_select_example
cd date_select_example
script/generate scaffold person name:string birthday:date
rake db:migrate

Now if we hit the site and try to create a new person we see a screen like this

And when you click "Create"

Exactly what you want with almost no code in the view and none in the model. I did make a minor edit to the view so we would get 1960 in the year select by adding :start_year=>1900 (a full documentation of date_select options are available here)

#app/views/people/new.html.erb
<h1>New person
<% form_for(@person) do |f| %>
  <%= f.error_messages %>
  <p>
    <%= f.label :name %><br />
    <%= f.text_field :name %>
  </p>
  <p>
    <%= f.label :birthday %><br />
    <%= f.date_select :birthday, :start_year=>1900 %>
  </p>
  <p>
    <%= f.submit 'Create' %>
  </p>
<% end %>

<%= link_to 'Back', people_path %>

and

#app/models/person.rb
class Person < ActiveRecord::Base
end

Errors when storing as text in the database

Now what happens when you run into a case where the date is stored as a string in the database. Let's say we have an "anniversary" attribute stored as a string that we want to treat the same way as we did birthday. We create the migration.

script/generate migration AddAnniversaryToPerson anniversary:string
rake db:migrate

Then add the anniversary to our view.

#app/views/people/edit.html.erb
<h1>Editing person
<% form_for(@person) do |f| %>
  <%= f.error_messages %>
  <p>
    <%= f.label :name %><br />
    <%= f.text_field :name %>
  </p>
  <p>
    <%= f.label :birthday %><br />
    <%= f.date_select :birthday, :start_year=>1900  %>
  </p>
  <p>
    <%= f.label :anniversary %><br />
    <%= f.date_select :anniversary, :start_year=>1900  %>
  </p>
  <p>
    <%= f.submit 'Update' %>
  </p>
<% end %>

<%= link_to 'Show', @person %> |
<%= link_to 'Back', people_path %>

It looks like we're done so we click "Update" and .. Oops. It doesn't work! We get the error 1 error(s) on assignment of multiparameter attributes. Now we need to figure out what multiparameter attributes are and whey they're not working for us.

ActiveRecord::MultiparameterAssignmentErrors in PeopleController#update

1 error(s) on assignment of multiparameter attributes
RAILS_ROOT: /Users/alexrothenberg/date_select_example

Application Trace | Framework Trace | Full Trace
/Library/Ruby/Gems/1.8/gems/activerecord-2.3.2/lib/active_record/base.rb:3061:in `execute_callstack_for_multiparameter_attributes'
/Library/Ruby/Gems/1.8/gems/activerecord-2.3.2/lib/active_record/base.rb:3022:in `assign_multiparameter_attributes'
/Library/Ruby/Gems/1.8/gems/activerecord-2.3.2/lib/active_record/base.rb:2749:in `attributes='
/Library/Ruby/Gems/1.8/gems/activerecord-2.3.2/lib/active_record/base.rb:2627:in `update_attributes'
/Users/alexrothenberg/date_select_example/app/controllers/people_controller.rb:63:in `update'
...more stack trace...

Request Parameters:

{"commit"=>"Update",
 "_method"=>"put",
 "authenticity_token"=>"qezkVq+MNzFuXxFBJ/GaSoh2BNdxM6oF3H7JP5beFFE=",
 "id"=>"1",
 "person"=>{"name"=>"Barack Obama",
 "birthday(2i)"=>"8",
 "birthday(3i)"=>"4",
 "anniversary(1i)"=>"2009",
 "anniversary(2i)"=>"5",
 "anniversary(3i)"=>"22",
 "birthday(1i)"=>"1960"}}

What happened? There are two keys to figuring out what's going on

The date_select helper actually sends 3 http parameters to our application anniversary(1i), anniversary(2i) and anniversary(3i). ActiveRecord must combine those into a single Date before updating the row in the database.
Looking at assign_multiparameter_attributes in active_record/base.rb we see this this comment that talks about combining 3 http parameters into a date type by calling new on the column type

Instantiates objects for all attribute classes that needs more than one constructor parameter. This is done by calling new on the column type or aggregation type (through composed_of) object with these parameters.
So having the pairs written_on(1) = "2004", written_on(2) = "6", written_on(3) = "24", will instantiate written_on (a date type) with Date.new("2004", "6", "24"). You can also specify a typecast character in the parentheses to have the parameters typecasted before they're used in the constructor. Use i for Fixnum, f for Float, s for String, and a for Array. If all the values for a given attribute are empty, the attribute will be set to nil.

Now we know we're close to the right place and can check the column's return type using script/console

  $ script/console
  Loading development environment (Rails 2.3.2)
  >> Person.columns_hash['birthday']
  => #
  >> Person.columns_hash['birthday'].klass
  => Date
  >> Person.columns_hash['anniversary']
  => #
  >> Person.columns_hash['anniversary'].klass
  => String

This makes sense. ActiveRecord is treating the anniversary column as a string because that's what it is in the database so is not combining the 3 multiparameter attributes into a Date. What we need to do is override the default ActiveRecord logic and tell it to treat this column as a date. We can write a failing spec

require File.dirname(__FILE__) + '/../spec_helper'

describe Person do
  it "should treat anniversary as a Date column" do
    Person.columns_hash['anniversary'].klass.should == Date
  end
end

# 'Person should treat anniversary as a Date column' FAILED
# expected: Date,
#      got: String (using ==)

Then take advantage of a trick of Ruby that allows you to extend an object without affecting other instances of its class (see PickAxe book's explanation of this technique). If we lookup the definition of klass for a column It is actually very simple to implement our fix with the 5 lines below.

class Person < ActiveRecord::Base
  class << columns_hash['anniversary']
    def type
      :date
    end
  end
end

Now our tests pass and when we go back to our site and click the Update button and it works.

Testing AJAX without a browser with Cucumber and Webrat

2009-05-06T00:00:00-07:00

I have lately fallen in love with using Cucumber and Webrat for my integration/acceptance testing. Cucumber because it allows non-technical people to write or at least read the test scenarios and Webrat because it matches content and encourages you to write integration tests without relying on xpath to find html elements. The way I like to use these tools is to run Rails integration tests which means its fast since I don’t need to start a mongrel or fire up a browser and can use Rails’ transactional fixtures to rollback all my database changes at the end of each test scenario. The only downside is that you can’t test javascript.

Today I am going to talk about how to get around this and test a form with an ajax autocomplete field. I've built a sample application with all the code examples here and you can download it from http://github.com/alexrothenberg/testing-ajax-example if you like. The application I'm building is just some simple app created with scaffolding that just has a User resource with a name and address. I modified the /users page to not display all users but include the auto_complete typeahead to let you pick a user (imagining there may be a lot) so the page looks something like this.

My first test scenario will ignore the ajax and just test the form which is super easy and can be done by writing a single feature file.

#features/find_a_user.feature
Feature: Allow anyone to find a user and see their details
  In order to handle a large set of users
  I want search with autocomplete
 
  Scenario: View a candidate detail page without testing ajax
    Given "Mickey Mouse" is a user living at "123 Main Street"
    When I am on the homepage
      And I fill in "Which user" with "Mickey Mouse"
      And I press "Find"
    Then I should see "Mickey Mouse"
      And I should see "123 Main Street"

I run it it all passes and I get

$ rake features
(in /Users/alexrothenberg/ruby/testing-ajax-example)
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby -I "/Library/Ruby/Gems/1.8/gems/cucumber-0.3.2/lib:lib" 
"/Library/Ruby/Gems/1.8/gems/cucumber-0.3.2/bin/cucumber" --format pretty --require features/step_definitions/user_steps.rb 
--require features/step_definitions/webrat_steps.rb --require features/support/env.rb --require features/support/paths.rb 
features/find_a_user.feature
Feature: Allow anyone to find a user and see their details
  In order to handle a large set of users 
  I want search with autocomplete

  Scenario: View a candidate detail page without testing ajax     # features/find_a_user.feature:5
    Given "Mickey Mouse" is a user living at "123 Main Street"    # features/step_definitions/user_steps.rb:1
    When I am on the homepage                                     # features/step_definitions/webrat_steps.rb:6
    And I fill in "Which user" with "Mickey Mouse"                # features/step_definitions/webrat_steps.rb:22
    And I press "Find"                                            # features/step_definitions/webrat_steps.rb:14
    Then I should see "Mickey Mouse"                              # features/step_definitions/webrat_steps.rb:93
    And I should see "123 Main Street"                            # features/step_definitions/webrat_steps.rb:93

1 scenario (1 passed)
6 steps (6 passed)

I makes use of the default webrat steps that cucumber gives you for free in features/steps/webrat_steps.rb so I don’t even have to write any code to get it to pass but I still haven’t tested any of my code that responds to the autocomplete request. I’d like my test to verify that my routes, controller and model will all work together. So, I write another scenario and run it and this time it fails because I haven't defined the typeahead steps for the typeahead lines.

#first scenario omitted

Scenario: View a candidate detail page testing (most of) the ajax # features/find_a_user.feature:13
    Given "Mickey Mouse" is a user living at "123 Main Street"      # features/step_definitions/user_steps.rb:1
    When I am on the homepage                                       # features/step_definitions/webrat_steps.rb:6
    And I typeahead in "Which user" with "Mickey Mouse"             # features/find_a_user.feature:16
    And I fill in "Which candidate" with the first typeahead result # features/find_a_user.feature:17
    And I press "Find"                                              # features/step_definitions/webrat_steps.rb:14
    Then I should see "Mickey Mouse"                                # features/step_definitions/webrat_steps.rb:93
    And I should see "123 Main Street"                              # features/step_definitions/webrat_steps.rb:93

2 scenarios (1 undefined, 1 passed)
13 steps (3 skipped, 2 undefined, 8 passed)

You can implement step definitions for undefined steps with these snippets:

When /^I typeahead in "([^\"]*)" with "([^\"]*)"$/ do |arg1, arg2|
  pending
end

When /^I fill in "([^\"]*)" with the first typeahead result$/ do |arg1|
  pending
end

Now I take the hints cucumber has given me and write my autocomplete steps. The interesting thing here is that I need to leave the response object unchanged so I can fill in the form field after running the typeahead step so I can't use the existing webrat steps as they work on a single pair of request and response objects. So I knew I'd be creating a new class with its own request and response that could be used without affecting the one used by my other cucumber steps. Using good outside-in development practices I deferred thinking about how to do that and first wrote my steps file to look something like this. One interesting thing to notice here is that you can invoke a step from inside another step just by omitting the block as I do in the second step.

When /^I typeahead in "(.*)" with "(.*)"$/ do |field, value|
  field = field_labeled field
  @typeahead = AutoCompleteStepHelper.new(request)
  @typeahead.type(field, value)
end

When /^I fill in "(.*)" with the first typeahead result$/ do |field|
  When %Q[I fill in "#{field}" with "#{@typeahead.items.first}"]
end

Now when run the feature again it fails telling me I haven’t yet built the AutoCompleteStepHelper.

#first scenario omitted

Scenario: View a candidate detail page testing (most of) the ajax # features/find_a_user.feature:13
    Given "Mickey Mouse" is a user living at "123 Main Street"      # features/step_definitions/user_steps.rb:1
    When I am on the homepage                                       # features/step_definitions/webrat_steps.rb:6
    And I typeahead in "Which user" with "Mickey Mouse"             # features/step_definitions/autocomplete_steps.rb:1
      uninitialized constant AutoCompleteStepHelper (NameError)
      ./features/step_definitions/autocomplete_steps.rb:3:in `/^I typeahead in "(.*)" with "(.*)"$/'
      features/find_a_user.feature:16:in `And I typeahead in "Which user" with "Mickey Mouse"'
    And I fill in "Which candidate" with the first typeahead result # features/step_definitions/autocomplete_steps.rb:7
    And I press "Find"                                              # features/step_definitions/webrat_steps.rb:14
    Then I should see "Mickey Mouse"                                # features/step_definitions/webrat_steps.rb:93
    And I should see "123 Main Street"                              # features/step_definitions/webrat_steps.rb:93

2 scenarios (1 failed, 1 passed)
13 steps (1 failed, 4 skipped, 8 passed)

So the next step is to write the AutoCompleteStepHelper class. This class’ job is to have its own request and response that will not affect the ones used by cucumber for the main page requests. It turns out that I can do this by having my class extend ActionController::IntegrationTest and I can even use webrat methods in it because webrat adds its methods to IntegrationTest. In this example I'm calling visit and current_dom and using nokogiri to parse the dom. It is a little weird that I'm subclassing IntegrationTest but this class is not a TestUnit class itself but I decided that was okay.

class AutoCompleteStepsHelper < ActionController::IntegrationTest
  def initialize(existing_request)
    @controller_name = existing_request.parameters[:controller]
    @controller_class = "#{@controller_name.to_s.camelize}Controller".constantize
    raise "Can't determine controller class for #{@controller_class_name}" if @controller_class.nil?

    @controller = @controller_class.new
    @request = ActionController::TestRequest.new
    @response = ActionController::TestResponse.new
    @response.session = @request.session
  end
 
  def type(field, value)
    visit url_for(:controller=>@controller_name, :action=>"auto_complete_for_#{field.id}", field.send(:name)=>value)
  end
 
  def items
    current_dom.search('//ul/li').map(&:inner_html)
  end
end

Now when I run the feature it all passes.

#first scenario omitted

  Scenario: View a candidate detail page testing (most of) the ajax # features/find_a_user.feature:13
    Given "Mickey Mouse" is a user living at "123 Main Street"      # features/step_definitions/user_steps.rb:1
    When I am on the homepage                                       # features/step_definitions/webrat_steps.rb:6
    And I typeahead in "Which user" with "Mickey Mouse"             # features/step_definitions/autocomplete_steps.rb:1
    And I fill in "Which user" with the first typeahead result      # features/step_definitions/autocomplete_steps.rb:7
    And I press "Find"                                              # features/step_definitions/webrat_steps.rb:14
    Then I should see "Mickey Mouse"                                # features/step_definitions/webrat_steps.rb:93
    And I should see "123 Main Street"                              # features/step_definitions/webrat_steps.rb:93

2 scenarios (2 passed)
13 steps (13 passed)

The last step I took was to test that the list returned in the typeahead was correct. I created another scenario in my feature.

Scenario: Typeahead should return 2 users that match but not a third
    Given "Mickey Mouse" is a user living at "123 Main Street"
      And "Donald Duck" is a user living at "123 Pond Lane"
      And "Minnie Mouse" is a user living at "123 Disney Avenue"
    When I am on the homepage
      And I typeahead in "Which user" with "Mi"
    Then I should see in my typeahead "Mickey Mouse"
      And I should see in my typeahead "Minnie Mouse"
      And I should not see in my typeahead "Donald Duck"

and I added two new steps

Then /^I should see in my typeahead "(.*)"$/ do |text|
  @typeahead.response_body.should =~ /#{text}/m
end

Then /^I should not see in my typeahead "(.*)"$/ do |text|
  @typeahead.response_body.should_not =~ /#{text}/m
end

Now when I run my features all 3 scenarios are passing

$ rake features
(in /Users/alexrothenberg/ruby/testing-ajax-example)
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby -I "/Library/Ruby/Gems/1.8/gems/cucumber-0.3.2/lib:lib" 
"/Library/Ruby/Gems/1.8/gems/cucumber-0.3.2/bin/cucumber" --format pretty --require features/step_definitions/autocomplete_steps.rb 
--require features/step_definitions/user_steps.rb --require features/step_definitions/webrat_steps.rb 
--require features/support/autocomplete_steps_helper.rb --require features/support/env.rb --require features/support/paths.rb 
features/find_a_user.feature

Feature: Allow anyone to find a user and see their details
  In order to handle a large set of users 
  I want search with autocomplete

  Scenario: View a candidate detail page without testing ajax  # features/find_a_user.feature:5
    Given "Mickey Mouse" is a user living at "123 Main Street" # features/step_definitions/user_steps.rb:1
    When I am on the homepage                                  # features/step_definitions/webrat_steps.rb:6
    And I fill in "Which user" with "Mickey Mouse"             # features/step_definitions/webrat_steps.rb:22
    And I press "Find"                                         # features/step_definitions/webrat_steps.rb:14
    Then I should see "Mickey Mouse"                           # features/step_definitions/webrat_steps.rb:93
    And I should see "123 Main Street"                         # features/step_definitions/webrat_steps.rb:93

  Scenario: View a candidate detail page testing (most of) the ajax # features/find_a_user.feature:13
    Given "Mickey Mouse" is a user living at "123 Main Street"      # features/step_definitions/user_steps.rb:1
    When I am on the homepage                                       # features/step_definitions/webrat_steps.rb:6
    And I typeahead in "Which user" with "Mickey Mouse"             # features/step_definitions/autocomplete_steps.rb:1
    And I fill in "Which user" with the first typeahead result      # features/step_definitions/autocomplete_steps.rb:7
    And I press "Find"                                              # features/step_definitions/webrat_steps.rb:14
    Then I should see "Mickey Mouse"                                # features/step_definitions/webrat_steps.rb:93
    And I should see "123 Main Street"                              # features/step_definitions/webrat_steps.rb:93

  Scenario: Typeahead should return 2 users that match but not a third # features/find_a_user.feature:22
    Given "Mickey Mouse" is a user living at "123 Main Street"         # features/step_definitions/user_steps.rb:1
    And "Donald Duck" is a user living at "123 Pond Lane"              # features/step_definitions/user_steps.rb:1
    And "Minnie Mouse" is a user living at "123 Disney Avenue"         # features/step_definitions/user_steps.rb:1
    When I am on the homepage                                          # features/step_definitions/webrat_steps.rb:6
    And I typeahead in "Which user" with "Mi"                          # features/step_definitions/autocomplete_steps.rb:1
    Then I should see in my typeahead "Mickey Mouse"                   # features/step_definitions/autocomplete_steps.rb:16
    And I should see in my typeahead "Minnie Mouse"                    # features/step_definitions/autocomplete_steps.rb:16
    And I should not see in my typeahead "Donald Duck"                 # features/step_definitions/autocomplete_steps.rb:20

3 scenarios (3 passed)
21 steps (21 passed)

What I've done is not full javascript testing (for that I'm planning to look into Blue-Ridge from Relevance). This technique does allow you to test ajax (skipping the "J") without a browser.

Managing the draw of a single-elimination tournament

2009-03-30T00:00:00-07:00

My company has been running a GO tournament and since its around NCAA March Madness time I thought it'd be easy to find a site to manage the draw. I looked but couldn't find anything that did what I wanted so I wrote my own. Caveat: I just spent a few days on this and it could stand to be enhanced with more features and a rewrite of the ui implementation but I'm open to suggestions if anyone finds this useful.

To use it

Edit the file lib/names.txt with a the players in your tournament

Run 'rake create_tournament

Now when you view the site you'll see a draw that looks something like this.

For games with 2 players - all first round games and games where the previous round winners have been determined - you'll be able to click the link and set which player won.

This was all very quickly thrown together but if you're interested in hosting any sort of single-elimination tournament check it out on github tournament-draw.

Let me know if you use it and how you'd like to enhance it from the quick application it is today.

Link: Search Engine in 200 lines of Ruby Code

2009-03-22T00:00:00-07:00

I read a very interesting article by someone who works for Yahoo about how he wrote a basic search engine in Ruby in just a few files. I wouldn't use this for a production system for as an example of how search engines work its fascinating and I plan to keep following his site to see how he enhances it

Description: http://blog.saush.com/2009/03/write-an-internet-search-engine-with-200-lines-of-ruby-code/

Source: http://github.com/sausheong/saushengine

... and he's got a really cool css/javascript formatter for the source in his blog. I've gotta learn more about SyntaxHighlighter

Using scopes in auto_complete plugin

2009-03-18T00:00:00-07:00

My colleague Pat Shaughnessy has spent a lot of time recently enhancing the auto_complete plugin. I suggest you read his blog posts and check out his fork of auto_complete on github to see the details.

I was reading his latest change to allow filtering of auto complete picklists and really like what he did but thought there was one thing that didn't quite feel right - the fact that you have to mix the application logic to filter the list with the plugin logic to find the list in the block in your controller.

Here's the code Pat wrote in his controller and what I'd like to avoid is having to re-implement the "LOWER(tasks.name) LIKE ?" portion that's already implemented in the filtered_auto_complete_for method of autocomplete.rb in the plugin.

# For task name auto complete, only display tasks
# that belong to the given project: 
filtered_auto_complete_for :task, :name do | find_options, params|
  find_options.merge!(
    {
      :include => :project,
      :conditions => [ "LOWER(tasks.name) LIKE ? AND projects.name = ?",
                       '%' + params['task']['name'].downcase + '%',
                       params['project'] ],
      :order => "tasks.name ASC"
    }
  )
end

I'd like to propose a slight modification so the block you write as part of your application can focus just on the filtering and leave the responsibility for the search with plugin. I'm also proposing we use scopes (which I don't think were around when the original auto_complete plugin was written) to filter and sort the list the plugin. Here is the code I'd like to write in my application.

class ProjectController < ApplicationController
  # For task name auto complete, only display tasks
  # that belong to the given project: 
  filtered_auto_complete_for :task, :name do | list, params |
    list.by_project(params['project']['id'])
  end
end

class Project < ActiveRecord::Base
  named_scope :by_project, 
              lambda {|project_id| {:conditions => {:project_id => project_id} } }
end

We can do this with a simple modification of the plugin implementation of filtered_auto_complete_for

def filtered_auto_complete_for(object, method)
  define_method("auto_complete_for_#{object}_#{method}") do
    find_options = { 
      :conditions => [ "LOWER(#{method}) LIKE ?", '%' +
        params[object][method].downcase + '%' ], 
      :order => "#{method} ASC",
      :limit => 10 }
    @items = object.to_s.camelize.constantize.scoped(find_options)
    @items = yield(@items, params) if block_given?

    render :inline => "<%= auto_complete_result @items, '#{method}' %>"
  end
end

Originally, it used passed the find_options hash to the block and then executed the search in one step as "@items = object.to_s.camelize.constantize.find(:all, find_options)". My change is to rely on ActiveRecord proxy objects to chain criteria together leaving the block independent of the criteria here in the plugin. The best part is you don't have to worry about performance as ActiveRecord will intelligently combine the criteria into a single SQL request. For example searching for tasks that start with 'tas' within project #7


 User Load (1.2ms)   SELECT * FROM `tasks` WHERE ((`tasks`.`project_id` = '7') AND (LOWER(name) LIKE '%tas%'))  ORDER BY name ASC LIMIT 10

Java as a Legacy Language

2009-03-18T00:00:00-07:00

I came across this article titled Java as Legacy Language today. As an ex-Java guy who is now committed to Ruby I was amused by the title but also think he makes a good point.

One thing is for sure: If you're in the software development business, don't cling to old ways of doing development. And also, don't get too carried away thinking that something like Scrum is going to be the Bandaid that fixes your agility problems, because it may turn out that your main problem is Java itself. Keep an open mind. Try new things. Be ready when the next disruption arrives, or you may find yourself without a chair when the music stops.

I don't think Java is "bad" but I do think it encourages you to write big, complicated applications to solve big, complicated problems. In contrast I find Rails encourages you to realize most of the apps we write are in fact fairly simple and do the same CRUD steps over and over. Having so much support for that built into the framework helps you think about what's "interesting" about your appplication which may be unique but is probably not big.

Add to this how easy it is to test in Rails - tools like RSpec & Shoulda, mocking tools, integration testing and more recently cucumber + webrat all make it easier to practice TDD than not to. While there are tools in Java, the fact that they're harder to use, I believe, means fewer developers will use them and these practices will remain less ingrained in that community.

Microsoft Office links causing InvalidAuthenticityToken in Rails

2009-03-04T00:00:00-08:00

I started receiving a lot of error notifications recently from my ExceptionNotfier plugin for an error with ActionController::InvalidAuthenticityToken. It turned out the error was occurring because one of my users was pasting a link to my app in an MS Office document and when Office sees the link it makes a request that Rails could not handle. Here I'll show you a simple fix you can use to avoid these errors with much credit going to an article at
Dealing with Microsoft Office Protocol Discovery in Rails
.

My execptions looked something like this (lots of boring details omitted)


A ActionController::InvalidAuthenticityToken occurred in events#1164:

  ActionController::InvalidAuthenticityToken
  [RAILS_ROOT]/vendor/rails/actionpack/lib/action_controller/request_forgery_protection.rb:86:in `verify_authenticity_token'

-------------------------------
Environment:
-------------------------------
  * HTTP_USER_AGENT        : Microsoft Data Access Internet Publishing Provider Protocol Discovery
  * REQUEST_METHOD         : OPTIONS

The problem is that Rails doesn't understand the method 'OPTIONS' (see rails/actionpack/lib/action_controller/routing.rb

#around line 270 of routing.rb
module ActionController
  module Routing
    HTTP_METHODS = [:get, :head, :post, :put, :delete]
  end
end

Fixing the problem is fairly simple. You insert a before_filter into your application controller to intercept and handle requests with the option method before the rails code realizes it can't handle the request.

class ApplicationController < ActionController::Base
  before_filter :options_for_microsoft_office_protocol_discovery

  ### Lots of code omitted

  def options_for_microsoft_office_protocol_discovery
    render :nothing => true, :status => 200 if request.method == :options
  end
end

Its also easy to write a simple spec in rspec to verify the behavior. There is one trick which is that rails/actionpack/lib/action_controller/test_process.rb defines helper methods for get, post, put, delete & head that we can't use so we need to call the underlying method directly but the signature for that underlying method changed with Rails 2.3 (commit) so depending what version you're using you'll need one of 2 flavors.

# Rails 2.3 and above 
it 'should not throw an exception on OPTIONS request (from ms office protocol discovery)' do
  process '/any/goofy/path', nil, nil, nil, 'OPTIONS'
  response.should be_success
end 

# Rails < 2.3 version
it 'should not throw an exception on OPTIONS request (from ms office protocol discovery)' do
  @request.env['REQUEST_METHOD'] = 'OPTIONS'
  process '/any/goofy/path'
  response.should be_success
end

You can also test from the command line using curl


curl -X OPTIONS http://localhost:3000/

Update: It turns out that Rails stores the acceptable methods in 2 different places actionpack/lib/action_controller/request.rb which does include all of get head put post delete options and also (see rails/actionpack/lib/action_controller/routing.rb which only includes :get, :head, :post, :put, :delete (options is missing). This means this fix will only work for OPTIONS requests and not any other type as ActionController::Request request_method will throw an exception before getting to the filter code above.

Investigating how Symbol to_proc works

2009-02-07T00:00:00-08:00

One of the things I love about Ruby is how expressive it is and how with open classes it can be optimized to become even more expressive. Since I started using Ruby I don't think I've written a single for or while loop - something I couldn't have imagined saying with any other language! Of course I do this by using iterators and writing code like

user_names = User.all.collect {|user| user.name }

I recently started discovered I could write the same thing even more concisely (as long as I'm using Rails or Ruby 1.9)

user_names = User.all.collect(&:name)

I decided to investigate how this works.

First I found some good posts by Prag Dave and Ryan Bates and at InfoQ. This helped but I still didn't understand it all so decided to dig further.

First I took a look at how Rails extends Symbol

unless :to_proc.respond_to?(:to_proc)
  class Symbol
    # Turns the symbol into a simple proc, which is especially useful for enumerations. Examples:
    #
    #   # The same as people.collect { |p| p.name }
    #   people.collect(&:name)
    #
    #   # The same as people.select { |p| p.manager? }.collect { |p| p.salary }
    #   people.select(&:manager?).collect(&:salary)
    def to_proc
      Proc.new { |*args| args.shift.__send__(self, *args) }
    end
  end
end

So they defined the to_proc method on symbol and that means the new code will be called when we write &:name because it magically gets transformed into :name.to_proc. I learned something but still needed to learn more to understand how it all works.

Why does the & cause Ruby to call to_proc? I knew that & in the last parameter declaration will pass a provided block as a parameter but this seems to be doing the reverse. Calling a method as an argument but having it interpreted as a block. I tried a couple of experiments in irb

def was_block_given?
  block_given?
end

# As expected
was_block_given? {}
=>  true 
# Passing a proc is not the same as having a block
was_block_given? Proc.new{}
ArgumentError: wrong number of arguments (1 for 0)
  from (irb):195:in 'was_block_given?'
  from (irb):195
  
# Prefixing the proc with an &amp; makes it like a block
was_block_given? &amp;Proc.new{}
=> true

It was not all as I expected but some reading through the PickAxe book led me to a better understanding. I found this paragraph in the Calling A Method section (page 115 in my copy)

If the last argument to a method is preceded by an ampersand, Ruby assumes that it is a Proc object.
It removes it from the parameter list, converts the Proc object into a block, and associates it with the method.

Ok so now I know why when Ruby sees User.all.collect(&:name) it invokes the collect method with name.to_proc as a block. Next, it was time to figure out why the code Rails put in the to_proc method worked. I took a look at the Rubinius implementation Enumerable

def collect
  ary = []
  if block_given?
    each { |o| ary << yield(o) }
  else
    each { |o| ary << o }
  end
  ary
end

Again I decided to experiment with irb to see what each part of the to_proc implementation was doing. First I redefined the Symbol to_proc again with a puts so I could confirm what was going on.

class Symbol
  # Turns the symbol into a simple proc, which is especially useful for enumerations. Examples:
  #
  #   # The same as people.collect { |p| p.name }
  #   people.collect(&:name)
  #
  #   # The same as people.select { |p| p.manager? }.collect { |p| p.salary }
  #   people.select(&:manager?).collect(&:salary)
  def to_proc
    Proc.new do |*args| 
      puts "to_proc args: #{args.inspect}"
      args_shift = args.shift
      puts "to_proc: #{args_shift.inspect}.__send__(#{self.inspect}, *#{args.inspect})"
      result = args_shift.__send__(self, *args) 
      puts "to_proc result: #{result.inspect}"
      result
    end
  end
end

# Make the call and see what happens
[1].collect(&:to_s)
# to_proc args: [1]
# to_proc: 1.__send__(:to_s, *[])
# to_proc result: "1"
# => ["1"]

Its brute force but tells us everything we need to know. As expected collect yields to our proc/block with the element in a variable length argument [1], it extracts the 1 and sends it the to_s method with no arguments returning the string "1". At this point I think I understand how it all works and decide to confirm by running a few more (more complicated) tests in irb

It all works as expected and I decide I know as much as I need to about this and call it a day.

So why did I bother figuring all this out and then writing it up? Mostly because I didn't know how it worked and thought there was some 'magic' going on. I could have continued using this feature without understanding how it worked but now that I understand how it works if some need ever arises for me to do some similar magic I know how to go about it. As for writing it up I hope someone else may read this and find it useful but by I increased my own understanding through the act of writing.

Maintaining your technical chops is a full time job

2009-02-05T00:00:00-08:00

Great quote from Uncle Bob (near the bottom of the post)

I think that maintaining your technical chops is a full time job. For that reason I have avoided becoming a business wonk. I hire people to do that for me so I can keep my technical skills as sharp as possible and remain relevant to my profession. I don’t believe I can offer technical advice unless I am living that technical advice.

Never sacrifice quality for speed!

2009-02-04T00:00:00-08:00

I just read two really good articles by Ron Jeffries and Uncle Bob about why sacrificing quality to go faster is always a bad idea.

This is very relevant to me now as I'm working with a sponsor now who thinks that by pushing harder and 'doing it in parallel' he can get everything he wants by the date he wants avoiding any hard decisions involving tradeoffs between scope and date. I have worked on many teams over the years that could speed up without sacrificing quality by focusing on the right things basically by emphasizing working software over high ceremony (pretty much straight from the Agile Manifesto). However the idea that you can get something out the door quickly without "wasting your time on quality" to me means that the person you're talking to doesn't understand quality. They think quality means those silly engineering things developers spend their time on rather than software that works the way you want and can be reliably enhanced later.

Somehow we as technologists need to do a better job of explaining to our business sponsors that "quality" is not a technical term!

How I refine a test spec while writing getting it to green

2009-01-09T00:00:00-08:00

In most of the examples I've read on TDD they show the Red-Green-Refactor cycle as

Write a failing test
Write just enough code to make the test pass
Refactor
Repeat

I absolutely do believe in this cycle and live it every day however I think there's a small detail that differs from what I do. When I move from step 1 to step 2, I keep my test window open and will switch back and forth between the test and code windows and refining the test while writing the code that makes the test pass. Its only as I write the code that I realize what calls I need to mock out in the test which also adds additional expectations to my test. I'm sure this is how most people work but I've never seen it written up so I'm going to try going through an example here.

Let's say I have a social networking application with a page showing friend requests where you can accept or reject each request. Something like this

Friend Requests:

Now I'm ready to implement an action in the Friend Controller that receives a posted form with a list of friend requests to accept and reject. I might first go into my friends_controller_spec.rb and write something like


describe FriendController do
  it 'should approve one friend requests' do
    post :update_requests, :approved_requests => ['123']
    response.should be_redirect
    response.should redirect_to(:back)
    flash[:notice].should  == "Friend requests approved."
  end
end

Now I have a failing test so I start implementing the code


class FriendController < ApplicationController
  def update_requests
    FriendRequest.approve(params[:approve_requests])
    flash[:notice] = "User access changes saved."
    redirect_to :back
  end
end

I'm done but my test is still failing...why? I haven't added the approve method to my FriendRequest model yet. I could go ahead and implement that method to get this test to pass but I don't want to do that because then this test would be testing more than just one unit. This is a unit test for the update_requests method of my FriendController and should be isolated from bugs in the rest of my application. Time to go back to the spec and add a mock to isolate this spec from the model and impose another expectation on my spec (that the method be called with the correct argument).


describe FriendController do
  it 'should approve one friend requests' do
    FriendRequest.expects(:approve).with(['123'])
    request.env['HTTP_REFERER'] = "http://some.site.com"
  
    post :update_requests, :approved_requests => ['123']
    response.should be_redirect
    response.should redirect_to(:back)
    flash[:notice].should  == "Friend requests approved."
  end
end

Now my spec passes and I can do my refactoring then go back to the beginning and write my next expectation.

The point of this example is that when I first wrote the spec I hadn't yet thought about how I would implement update_requests so did not yet know that I would need to mock FriendRequest.approve. I do this all the time and typically find myself go back and forth between the code and spec many times with the spec telling me what code I need to write and the code telling me how to refine the spec.

One last point (and a teaser for a future post). You have probably noticed that although my spec passes if I try to run the application it will not work because I still haven't written the FriendRequest.approve method. In a future post I hope to discuss how to interleave integration testing into this process to ensure that the classes we're writing in isolation also integrate so that the application satisfies its business function.

How I use TDD for Integration and Isolation testing

2008-12-11T00:00:00-08:00

I read an interesting post today by David Chelimsky who wrote RSpec A case against a case against mocking and stubbing. Its about mocking in testing and isolated vs integrated tests. I liked it all but what I particularly liked is how it describes the process of outside-in development.

To quote:

Write scenarios in plain text with cucumber (driven by user stories, organized in features).
Write the code for a step (or part of a step), run the feature, and observe the failure.
Optionally (yes, it depends – and why is the topic for another blog) drive out a view with a view spec. When I say “drive out,” I mean a very granular Red/Green/Refactor cycle that only involves this view, and only enough of this view to support satisfaction of the failing step in the cucumber feature.
Drive out a controller action using the same, granular Red/Green/Refactor cycle. And it may not be the entire controller action I think I want, covering all the cases I think I want. Just enough to support satisfaction of the failing step.
Drive out the parts of the model that I need to satisfy the failing step, using the same granular R/G/R process.
Run the cucumber feature and assess where I am.

In my projects I think I'm pretty good at following 3,4 &5 but I've struggled with steps 1 & 6. With the advent of cucumber I think now is the time to work on those other steps. I'll let you know how it goes in the New Year.

Knows which specs are slowing down my spec suite

2008-12-06T00:00:00-08:00

I often watch my specs run with the ........'s marching across my terminal and sometimes it seems to pause as it hits a particularly slow one. I figure there's some poorly written spec that's slowing my entire suite down and I should figure out what it is and fix it. But I rarely (if ever) take the time to track down the offender so never fix it. I just discovered that the clever folks writing rspec have taken away my excuse for laziness. I can tell it to report on the slowest specs by putting a line in my spec.opts file telling it to use the ProfileFormatter (see the first line below)


--format profile
--colour
--loadby mtime
--reverse

For example running the specs on my current project I get this output.


Profiling enabled.
........(a bunch more dots)

Top 10 slowest examples:
2.4908950 WikiContent should find all activity for a user
0.8061750 WikiContent should find recent activity for a user in last 15 days
0.2438460 WikiContent::Version should allow limit on results from finding recent activity
0.1821710 User should have a list of owned content
0.1700670 WikiContent should find results if access type is not same as role of the user but role of user is 'Leader'
0.1593900 ContentsController allows PUT request to 'update' action for 'member' role
0.1514150 Search::SearchResponse gets the wiki content id of an asset from the xml response
0.1470310 Content should always allow leaders to see all pages
0.1394920 Page should delete all comments when destroyed
0.1370330 ContentsController should save content and create the uploaded asset on successful POST to the 'create' action

Over the next few days I will find time to take a look at 'WikiContent find all activity' and 'WikiContent find recent activity' specs as they are the two clear outliers.

I now have no excuse to not optimize my slowest specs because I didn't know which they were!

Bug/patch with rspec-rails and helper instance variables

2008-12-06T00:00:00-08:00

I finally got around to upgrading my version of rspec-rails from one that's almost a year old and came across an issue with the way implicit module inclusion is handled.

If you have a handler that uses memoization to cache some information in instance variables such as this (I'm not sure if this is a smell but my project has some examples of it)


module UsersHelper
 def all_users
   @users ||= User.find(:all)
 end
end

You would expect this spec to work


describe UsersHelper do
 it "should find all users" do
   User.expects(:find).with(:all).returns(result=mock)
   helper.all_users.should == result
 end
end

Sometimes it does but if any other spec has called helper.all_users previously it will fail as the memoized @users variable is not nil so the collection is reused.

I have submitted a lighthouse patch to rspec-rails that fixes the problem so hopefully they will agree to fix it soon. Until then patch is available on github in my fork or rspec-rails if you want to use it.

Applications usually use their own data

2008-11-25T00:00:00-08:00

I just read a very interesting post by Martin Fowler called Database Thaw. He talks about various database technologies and what the future might hold for object or relational databases but what caught my interest was his discussion of application and database integration patterns.

He says "For many organizations today, the primary pattern for integration is Shared Database Integration - where multiple applications are integrated by all using a common database." This is certainly true where I work. I've even heard it said that we have one gigantic application since all (or most) share the same underlying database so cannot be updated independently.

He also talks about Integration Databases which store data for multiple applications and Application Databases which are controlled and accessed by a single application.

Historically my company has moved toward the integration database pattern often building complex service layers to enable shared access to the data on the assumption that if you build it they will come the data would be reused if it was easy enough. However I believe that its only a very small subset of our data that is truly common and many applications want to use. For that small subset a shared database with a service layer is probably a good design. The vast majority is private to an application and by designing it into a separate application database can evolve to best meet the needs of the application without concern for a more general service that in our case is more hindrance than benefit.

One important thing to remember about an application database is that the data exists to meet the needs of the application so a design that allows the database to best meet the application's needs will be to everyone's benefit. Finally database technologies are very complex and do a lot so it is probably wise to have a good database developer on your team to make sure you're using the database appropriately but they should be a part of your team not on a separate database team as I've seen in the past - particularly on Java/Oracle projects I've been a part of.

Thoughts?

Silly site - Netdisaster ... Destroy the Web

2008-11-20T00:00:00-08:00

My kids just showed me a silly site http://www.netdisaster.com/ I had to share.

Buy services and Build websites

2008-11-16T00:00:00-08:00

Where I work there's recently been a lot of talk about how to make the decision whether to buy or build software. I've recently concluded that while you can buy a commoditized service you can not buy a website!

What I've seen is people thinking we could "buy" (or download open source) a site but then realize they want to change the branding, adapt the terminology to match our business. combine it with another product or add one extra piece of functionality that the existing site doesn't have. Whatever the APIs provided by the product you're left with custom software written on top of the product you bought. I've seen this with big commercial products like Documentum and open source products like Drupal. I now believe that there's no such thing as "a little" customization and you can only buy a site if you want 0 customization (applying a skin or inserting your logos are acceptable).

This does not mean everyone should develop custom software from scratch. There are tons of opportunities to buy commoditized services and write just a little custom code to glue them together. This allows you to focus on the aspects of your site that are business differentiators (the branding, the terminology, your workflows, your business rules, etc) without fighting the vendor's idea of those things. You don't have to write too much code as what we continually revise upward what we consider a commoditized service (i.e. it used to be document storage and is becoming document management with workflow and security).

The process of creating a blog aggregator as I described in my last post I built a Blog Aggregator showed me how true this is. I used Atom and RSS parsing and generating service which left me just to build a small site with the UI, security and glue holding it all together. Its just a small example but as I look around github I'm struck that the most interesting packages are plugins and gems for others to use and the less interesting ones are complete sites. This may be a no brainer for the Open Source community but is something that I am just realizing and I think that enterprises and commercial vendors still need to learn.

Alex Rothenberg

Programming With Kids

Shoes

Gosu

gamesalad

What’s next?

Twitter is the new RSS Reader

Enabling API access to twitter

Updating WayWeWork.IT to tweet new posts

Using BDD and the email_spec gem to implement Email

Our Sample Project

Adding the email_spec gem

Our first Cucumber Scenario

Dropping into RSpec unit tests

Checking the Cucumber Scenario…

Back down to the unit tests

We’re Done

Using Letter Opener to View Sent Email on a Server (without actually sending anything)

An example

Logfile testing (we can do better)

Using letter_opener in development

How does letter_opener actually work?

Using letter_opener on staging

Chaining ActiveRecord Scopes from Different Models

Adding Scopes to make it usable

Ammeter: The Way to Write Specs for Your Rails Generators

Who writes generators?

Writing Specs with Ammeter

Why ‘Ammeter’?

Feedback Welcome

Upgrading An Old Rails Rails 2.1.1 to 3.1 on Heroku

Create a new Rails 3.1 project

Add gems from my app

Add our existing code and specs

Update our code

Changing named_scope to scope

RAILS_ROOT => Rails.root

Routes.rb

RSpec Routing Specs

RSpec stub!

View Specs

Deploying to Heroku

Talk Ruby to a Ruby Class instead of JSON to an HTTP Service

An Example: User Directory Service

Building our client gem

Building a fake service

Running a Private GemServer inside the Firewall

Setup an inside-the-firewall gem server

Configure our gems to deploy to the gem server

Using your Gem Server from an application

How Bundler Groups relate to the Rails Environment

How Bundler installs gems into your bundle

How Rails and Bundler load your gems according to the Rails Environment

What if you create a gem group that doesn’t correspond to any Rails env?

Backbone.js Makes Building JavaScript Applications Fun

What is CodeBuddy

Organizing our data in Backbone Models

Building the UI with Backbone Views

Showing code for the selected address

Changing the selected address

Conclusion

Moved Blog to Jekyll and GitHub Pages

Why did I switch?

How did I create this new site?

1. Create an empty site

2. Get my existing content

3. Syntax highlighting code

4. Disqus comments

5. Mapping my domain name

Authoring and Publishing

Setting up a new Mac for Rails Development

Combining two related fields in a url to prevent tampering

Purging old cruise builds

Use dashes not underscores in your domain name

Its not too late to solve a Y2K bug - I found one in 2010

What to do when ActiveRecord thinks an Oracle key is a decimal

Freezing a gem that has native extensions

Generate Models from Tables - Legacy Data Gem

How to use it

Examples

Changing `named_scope` to `scope`

RSpec `stub!`