Using Cucumber’s Table Transformations, you can easily build complex objects in a way that’s easy to read and understand for clients and developers alike.

My latest favorite feature of Cucumber are Table Transformations. I frequently use tables to build up complex objects and I’ve found that the regular old tables can be a little ugly, especially when your attribute names don’t make much sense on their own. I’ve also noticed that building up associations can be a little wonky, usually requiring more steps than seem necessary.

Conventional Table Usage

Let’s look at an example of how we could use a table, without transformations, to build up some objects for our scenario.

1
2
3
4
5

Scenario: Editing a Spirit
  Given I have a spirit with the following attributes:
    | spirit_type    | country_of_origin | age | brand        | lgcy_prod_sku | name       |
    | Scotch Whisky  | Scotland          | 12  | The Balvenie | SC38181       | DoubleWood |
    | Scotch Whiskey | Scotland          | 12  | The Macallan | SC38245       |            |

1
2
3
4
5

Given /^I have a spirit with the following attributes:$/ do |table|
  table.hashes.each do |attributes|
    Spirit.create!(attributes)
  end
end

Be sure to use create! in your tests to prevent false positives (Thanks Aslak!)

Now this is all fairly simple, and it looks pretty easy to implement, but I see some problems. First, what if you were actually friends with your DBA (bear with me) and you knew better than to have an attribute in your model like country_of_origin or spirit_type. Chances are those are going to be used by many other records and should be pulled out and made into their own Models, Country and SpiritType respectively.¹

So what does our scenario look like with those two new models?

1
2
3
4
5
6
7
8
9
10
11

Scenario: Editing a Spirit
  Given I have a country with the following attributes:
    | name     | continent |
    | Scotland | Europe    |
  And I have a spirit type with the following attributes:
    | name           |
    | Scotch Whiskey |
  And I have a spirit with the following attributes:
    | age | brand        | lgcy_prod_sku | name       |
    | 12  | The Balvenie | SC38181       | DoubleWood |
    | 12  | The Macallan | SC38245       |            |

It’s a little more complex, for sure, but it’s not totally unmanageable. However, the key part that’s missing is how to link the two spirits with their spirit types and countries of origin.

You could add some more steps, but then you’ve got a conjunction step which is inflexible and brittle.

1

  And the spirit named "The Balvenie" is from "Scotland" and is a "Scotch Whiskey"

You could go back to your original step, and try to do some behind-the-scenes stuff to map country_of_origin to the correct country_id, but that gets messy too.

Transform Your Tables

The first step to making good use of table transformations is to make your tables more readable. Start by change the header row of your table to use meaningful representations of the real attribute names.

1
2
3
4

Given I have a spirit with the following attributes:
  | Spirit Type    | Country  | Age | Brand        | Legacy Product Code | Name       |
  | Scotch Whisky  | Scotland | 12  | The Balvenie | SC38181             | DoubleWood |
  | Scotch Whiskey | Scotland | 12  | The Macallan | SC38245             |            |

We’ve turned that weird lgcy_prod_sku attribute into something that your Product Owner can make sense of and we’ve be able to add Spirit Type and Country of Origin back to the table. Now let’s look at the transformation that makes this all work.

1
2
3
4
5
6
7
8
9
10
11
12

Transform /^table:Spirit Type,Country,Age,Brand,Legacy Product Code,Name$/ do |table|
  table.hashes.map do |hash|
    spirit_type = SpiritType.create!({:name => hash["Spirit Type"]})
    country = Country.create!({:name => hash["Country"]})
    spirit = Spirit.create!({:age => hash["Age"],
                            :brand => hash["Brand"],
                            :lgcy_prod_sku => hash["Legacy Product Code"],
                            :name => hash["Name"]})
 
    {:spirit_type => spirit_type, :country => country, :spirit => spirit}
  end
end

The transformation step definition looks a lot like a regular table step definition. There is a regular expression, like anything else in Cucumber, that has the same values as the header row in the table from our scenario. Just like the table step definition we have a table object which is just an array of hashes. We can go through each hash, do the actual transformation, and then return something to our table step definition. We are using map (same as collect) to return an array of hashes, which is just what the table step definition is expecting.

You will also see that we’re creating three different records, which we are returning in the hash we create at the end. Let’s go through those step-by-step:

1. Create a spirit type object from the hash["Spirit Type"] value
2. Create a country object from the hash["Country"] value
3. Create a spirit object from the several related hash values
4. Put all of our created objects into a hash

While we’ve created the objects, we still need to create the associations. I like to leave this for the table step definition rather than the transformation since I think it’s more obvious what’s going on with the values when you’re viewing the table step.

1
2
3
4
5
6
7

Given /^I have a spirit with the following attributes:$/ do |table|
  table.each do |group|
    spirit = group[:sprit]
    associations = {:country => group[:country], :spirit_type => group[:spirit_type]}
    spirit.update_attributes(associations)
  end
end

There are at least a dozen ways to get the spirit associated with the country and spirit type, so don’t feel like you have to follow this pattern every time. Since we’ve sent our table an array of hashes we can iterate over each hash, group, and work with the individual rows. Here’s how:

1. Extract the spirit object from the hash
2. Create another hash with the country and spirit type that Rails can make sense of
3. Use update_attributes to update the spirit object with the new associations

Transformation Tradeoffs

We’ve been able to take our original multi-step scenario and simplify it to a single step. We are using the proper place, the step definitions, to do the associations and we have made our scenario much easier to read for non-developers working on the project. But what did we give up?

The biggest issue I’ve found with using table transformations is that they can be inflexible when you need to add more attributes to your dynamically created object. If you are writing features, using your table to setup objects and then realize that you need to add another attribute, you’re going to have to edit your table transformation step and how you create objects from the hash. When you take this a step further and try to have two different table definitions, you’ll be looking at having two nearly identical table transformations.²

If you’re not already using regular old Cucumber tables to create objects, use this guide to get started. If you are using Cucumber tables to create objects, try to re-factor one of your scenarios and use the table transformation strategy. Once you start using Cucumber tables and table transformations you’ll instantly improve the readability, portability and efficiency of your steps.

¹ Ignore for now the issues with spirit_type and Rails Single Table Inheritance

² I’m guessing that there is some way you can get around this with regular expressions and to have more flexible transformation table steps, but I haven’t tried it yet.

There’s a common mistake that many software developers make when estimating projects. Here’s how you can avoid falling into this trap.

When estimating a project using the Planning Poker method, many developers like to use a baseline estimate for a given task. For example, many developers use CRUD, the creating, displaying, editing and deleting of a Model as their baseline estimate. Once they’ve got their baseline in mind, it makes it easier to estimate other stories that are more domain specific, or so it seems.

Baseline Estimates are Broken

When you’re using a task like CRUD as a baseline for your estimations, you can easily skew the estimations of the other stories in the project. Let’s say we’re using a 3 point baseline for CRUD stories.

1. As a user I should be able to upload a profile photo – 2
2. As a user I should be able to CRUD movies I’ve seen – 3
3. As a user I should be able to send a private message to another user – 5
4. As a user I should be able to create a trivia quiz for a movie that I’ve seen – 8

In this first example, the stories are probably estimated fairly well and compared to each other, the complexity is quite relative. What happens if we add a few more easy stories or a few more difficult stories?

1. As a user I should be able to see a contact e-mail on the home page – 1
2. As a user I should be able download the menu as a PDF - 1
3. As a user I should be able to read a privacy policy – 2
4. As a user I should be able to CRUD restaurant reviews – 3

With easier stories added, the CRUD story is definitely the most complex, but compared to the others it is significantly more complex than seeing a link on a page or viewing some text.

1. As a user I should be able to CRUD portfolio photos – 3
2. As a user I should be able to signup for a paid recurring account – 8
3. As a user I should be able to make connections with other users via Facebook – 13
4. As a user I should be able to send a rocket to the moon – !

When the other stories become much more complex, the CRUD task is again shown to be significantly different in complexity, this time in the other direction. In this group of stories its hard to imagine that managing portfolio photos is only two orders of magnitude away from a recurring payment e-commerce system.

Relative Complexity Works

Its important to estimate a group of stories so that the complexity of each story is relative to the next. In Mike Cohn’s Agile Estimating and Planning, he describes a method of estimating stories called “Analogy”.

When estimating by analogy, the estimator compares the story being estimated with one or more other stories. If the story is twice the size, it is given an estimate twice as large.

When you apply this technique to your estimation process, you will have a more coherent set of estimates. From this you will be more likely to determine an accurate estimated velocity and you will have a better overall sense for the scope of the project.

Hurdles to Estimating by Analogy

• Estimating all of your stories one by one makes it difficult to estimate a relative complexity as you only have the previously estimated stories with which to compare your current story. A group of especially simple or complex stories could be waiting towards the end of the sessions.
• Estimators who are new to the agile estimation process might have difficultly estimating a story without a point of reference.
• Estimating with a baseline can be a difficult habit to break since it provides a convenience and familiarity to the estimator.
• Two seemingly similar projects may have a greater than expected difference in total number of story points, even though their velocities are relatively the same.

With a little hard work and my secret development ingredient, you can be a better Agile Developer

Recently my fellow developers at Integrum and I took a survey that helped us assess our team with regard to our Agile practices. When taking the survey, and now reviewing it later on, I was struck by how many of the questions were related to a single concept. Many of the problem areas that can be uncovered by the survey, along with examples of one’s successes, come back to this one theme.

Are programmers nearly always confident that the code they’ve written recently does what it’s intended to do?

• Is there more than one bug per month in the business logic of completed stories?
• Can any programmer on the team currently build and test the software, and get unambiguous success / fail result, using a single command?
• When a programmer gets the latest code, is he nearly always confident that it will build successfully and pass all tests?
• Are fewer than five bugs per month discovered in the teamʼs finished work?
• After a line item is marked “complete” do team members later perform unexpected additional work, such as bug fixes or release polish, to finish it?
• Are programmers nearly always confident that the code they’ve written recently does what it’s intended to do?
• Are all programmers comfortable making changes to the code?
• Do programmers have more than one debug session per week that exceeds 10 minutes?
• Do unexpected design changes require difficult or costly changes to existing code?
• Do any programmers optimize code without conducting performance tests first?
• Is there more than one bug per month in the business logic of completed stories?
• Are any team members unsure about the quality of the software the team is producing?

What’s the common theme among these stories, and the secret to better agile development? Testing, testing and more testing.

The negative outcomes implied by some of these questions can be solved by testing. Spending time fixing “completed” stories? Probably something you could have tested. Conversely, the positive benefits implied by other questions can be had via testing. Want to make your code more inviting and easier to deal with for new team members or people unfamiliar with the project? Give them robust and well-written tests.

Estimating stories for an upcoming project is one of the more difficult tasks that agile teams have to perform. It’s never easy to determine how difficult it will be to implement a particular feature, especially when you’ve got different personalities, goals, and levels or experience in the same room. Unfortunately this all leads to people coming up with excuses and roadblocks which lead to inaccurate estimates.

I’ve identified seven problems pitfalls of the agile estimation process that I’m sure many other teams have experienced:

1) Estimating Time Rather Than Complexity

The point of estimating stories with planning poker cards is that you estimate based on the story’s complexity, not on how long it will take you to complete the actual feature. A story that’s an 8 is more complex than one that’s a 5, but it doesn’t meant that the 8 will take two days. It could take an hour or a week, it’s all relative.

Why it’s Bullshit
Estimates based on time make planning commitments difficult and velocities unreliable.

2) Not Estimating For “Simplest Possible Solution”

The “Simplest Possible Solution” is just that, what’s the simplest way that the feature described in the story can be implemented. When you get away from this you start going down all sorts of “what if” roads that always end up bumping up your estimate.

Why it’s Bullshit
Trying to guess what the product owner will want or what the completed feature entails is a waste of time.

3) I’ve Never Done That Before

There aren’t many software problems that haven’t been solved. Most of them are things that have been solved a thousand times in a hundred different ways. Adding complexity to a story because you’ve personally never solved that problem is shortsighted.

Why it’s Bullshit
Just because you’ve never done something doesn’t mean it’s complex. Lean on your team or network of developer peers to help solve these problems.

4) Estimating Stories In Excessive or No Isolation

Stories should be estimated in isolation, just as they should be written so as not to depend heavily on one another. However, developers will often try to assign too much complexity to a story because of assumed tasks or features that they think would accompany the story in a completed state. For example:

As a user I should be able to login
As a user I should be able to upload a profile photo
As a user I should be able to change my address

Some developers will see the first story and immediately think of the complexity of creating a user model, the controllers and views that go along with the entire registration process.

Alternatively some developers will see the last story and think “Oh I just need a form field for the e-mail address when the user is editing their profile.”

Why it’s Bullshit
The first extreme gives you chunks of related stories with too much padding that are never as complex individually as they are as a whole. The latter only (sometimes) works when the actual stories are extracted out into a bunch of very small stories, which has its own set of problems.

5) Gaming Velocities

If you’re looking to “guesstimate” how long a project will take to complete, you could grab some story cards and pick out what you think might be a week of work. If you add up the points assigned to those stories you’d have an estimated velocity. However, if you’ve padded your stories, or purposefully pick out a small number of stories, your project is going to appear much lengthier than it really is.

Why it’s Bullshit
You don’t look any better completing 50 points per iteration when you padded the hell out of your estimates than you do when you do 20 points per iteration with accurate estimates.

6) Always Assume The Worst!

There seems to be this mantra with some developers, “Always assume the worst!” When you come across a slightly vague story, let your imagination run wild and assume that the product owner is going to want the most complex solution possible.

Why it’s Bullshit
Remember, every story is a negotiation. You’re not going to know the exact details of the story until you have your planning meeting with the product owner. Often times the product owner would never have been able to dream up the solution on which you based your estimate.

7) Padding Padding Padding

Padding is all Bullshit
The problem of padding estimates creeps into nearly all of the above six issues. It introduces bad data early in the life of the project and makes every other step of the process unreliable. It’s almost always in an effort to cover one’s ass but it’s painfully transparent and reeks of amateurism.

Don’t pad your estimates.

I had a blast presenting at the recent Intel Developer Ignite. In my quick five minute presentation I mapped ten of Aesop’s fables to modern day software engineering challenges and principles. If you missed the event, or just want to check out my presentation again, here it is!

Age-Old Solutions to Everyday Problems

Big thanks to Intel and everyone involved for putting on a great event, I really enjoyed it and can’t wait to do it again soon.

Today I gave a presentation at Desert Code Camp about BDD, Cucumber, Webrat and User Stories. If you’d like to find review the slides or download the code I used during the presentation here they are.

Slides and Code

Desert Code Camp BDD Presentation Slides

http://github.com/clayton/desert-code-camp

If you’re using Cucumber and you’re not using Transformations you’re doing it wrong. I just started using these recently and ran into a problem with creating records using factory_girl factories that made use of sequences. While trying to create multiple Authlogic user records with a unique email and unique single_access_token using a Cucumber table, the functionality of Factory.next(:email) wasn’t working correctly, I would always get the same e-mail address. Turns out it was an easy fix, just had to use lazy attributes in my factory.

The Scenario, Step Definition and Factory

1
2
3
4
5
6
7
8
9
10
11
12

  Scenario: Presenter List
    Given the following presenters:
      | Name    | Bio                 | Website              |
      | Clayton | Rails dev @integrum | http://claytonlz.com |
      | Chris   | Scrum @integrum     |                      |
    And I am on the homepage
    When I follow "Presenters"
    Then I should see "Clayton"
    And I should see "Rails dev @integrum"
    And I should see "http://claytonlz.com"
    Then I should see "Chris"
    And I should see "Scrum @integrum"

My Step Definition
This uses the Transformation table below:

1
2
3
4
5

Given /^the following presenters:$/ do |table|
  table.each do |attrs|
    Factory.create(:user, attrs)
  end
end

This transformation takes a table like the one in my scenario above, and assigns the values to a hash using the actual model attribute names (Name isn’t an attribute on a user but name is). The regular cucumber step definition “consumes” this hash for each entry in the table and passes it to a Factory for creation.

1
2
3
4
5

Transform /^table:Name,Bio,Website$/ do |table|
  table.hashes.map do |hash|
    {:name => hash[:Name], :bio => hash[:Bio], :website => hash[:Website]}
  end
end

My Factory

This is a pretty basic factory for an authlogic user model, I’m using factory_girl sequences to give me a “unique” e-mail and single access token, which are required by Authlogic.

1
2
3
4
5
6
7
8
9

Factory.define :user do |user|
  user.email Factory.next(:email)
  user.name ""
  user.bio ""
  user.website ""
  user.password "password"
  user.password_confirmation "password"
  user.single_access_token Factory.next(:single_access_token)
end

The problem

The above scenario will fail when it tries to create the user records via the factories. You’ll see a validation error about how the user model requires a unique e-mail and single access token. You’ll be wondering, “hey why are my sequences working?”. When you inspect the log you’ll see that they are in fact NOT working.

The Quick Answer

The easy answer to this is that you need to use lazy attributes in your factory for the sequences so that they are loaded each time instead of once.

1
2
3
4
5
6
7
8
9

Factory.define :user do |user|
  user.email { Factory.next(:email) }
  user.name ""
  user.bio ""
  user.website ""
  user.password "password"
  user.password_confirmation "password"
  user.single_access_token { Factory.next(:single_access_token) }
end

Notice the curly braces around the sequences

The Longer Answer

The cucumber rdoc explains the Transform functionality, albeit somewhat hard to understand.

Registers a proc that will be called with a step definition argument if it matches the pattern passed as the first argument to Transform. Alternatively, if the pattern contains captures then they will be yielded as arguments to the provided proc. The return value of the proc is consequently yielded to the step definition.

I think the issue comes from something with the way these Procs are created, called and also their scope with regard to the step definition etc. I don’t think my ruby-fu is strong enough to give a good explanation but maybe I’m going in the right direction.

If you’ve followed my version of the authlogic account activation tutorial or the original version by Matt Hooks you might have run into this error:

When authlogic sends e-mails with the account activation link, it uses a url_for helper to build that link. Because the “Notifier” mailer is an instance of ActionMailer::Base and not ActionController::Base it doesn’t know what the host parameter of the URL should be, so you have to tell it explicitly.

Put the following into your environments/development.rb and environments/test.rb:

1
2

# This assumes you're running your local development server on port 3000 via script/server
config.action_mailer.default_url_options = { :host => "127.0.0.1:3000" }

Put this into your environments/production.rb:

1
2

# Replace example.org with your actual domain name
config.action_mailer.default_url_options = { :host => "example.org" }

I’ve been troubleshooting a problem on an existing application for the last week or so that deals with the parsing of dates using the Chronic time parsing library. Today the problem magically solved itself, without me doing anything. These types of self solving problems are usually more frustrating than problems you can’t solve at all, so I took a little extra time to experiment with the particular date format I was using and found what might be a problem with the Chronic library when it gets to the “fall back” DST change in the fall.

Do You Observe DST?

It looks like if your local environment is set to a timezone which observes DST, like Eastern Standard Time for instance, you get a chunk of dates around the switch in the fall where Chronic returns nil instead of the correct date. If you’re on a machine where the timezone is set to an area which does not observe DST, like Arizona, you won’t be able to replicate this problem.

Replicating the problem

Install the latest version of Chronic.

$> sudo gem install chronic

Fire up irb and give the following a shot.

1
2
3
4

require 'chronic'
(Date.parse("2009-01-01")..Date.parse("2009-12-31")).each do |d|
  puts Chronic.parse("next tuesday 6am", :now => d)
end

With ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux] on Ubuntu and my timezone set to EST/New York. I see a bunch of Tuesdays and then a blank section around the end of October / beginning of November.

...snip...
Tue Oct 20 06:00:00 -0400 2009
Tue Oct 20 06:00:00 -0400 2009
Tue Oct 27 06:00:00 -0400 2009
Tue Oct 27 06:00:00 -0400 2009
Tue Oct 27 06:00:00 -0400 2009
Tue Oct 27 06:00:00 -0400 2009
Tue Oct 27 06:00:00 -0400 2009
Tue Oct 27 06:00:00 -0400 2009
Tue Oct 27 06:00:00 -0400 2009
nil
nil
nil
nil
nil
nil
Tue Nov 03 06:00:00 -0500 2009
Tue Nov 10 06:00:00 -0500 2009
Tue Nov 10 06:00:00 -0500 2009
Tue Nov 10 06:00:00 -0500 2009
Tue Nov 10 06:00:00 -0500 2009
Tue Nov 10 06:00:00 -0500 2009
Tue Nov 10 06:00:00 -0500 2009
Tue Nov 10 06:00:00 -0500 2009
Tue Nov 17 06:00:00 -0500 2009
Tue Nov 17 06:00:00 -0500 2009
Tue Nov 17 06:00:00 -0500 2009
...snip...

If you change your timezone to something like MST/Arizona you’ll see this

...snip...
Tue Oct 20 06:00:00 -0400 2009
Tue Oct 20 06:00:00 -0400 2009
Tue Oct 27 06:00:00 -0400 2009
Tue Oct 27 06:00:00 -0400 2009
Tue Oct 27 06:00:00 -0400 2009
Tue Oct 27 06:00:00 -0400 2009
Tue Oct 27 06:00:00 -0400 2009
Tue Oct 27 06:00:00 -0400 2009
Tue Oct 27 06:00:00 -0400 2009
Tue Nov 03 06:00:00 -0500 2009
Tue Nov 03 06:00:00 -0500 2009
Tue Nov 03 06:00:00 -0500 2009
Tue Nov 03 06:00:00 -0500 2009
Tue Nov 03 06:00:00 -0500 2009
Tue Nov 03 06:00:00 -0500 2009
Tue Nov 03 06:00:00 -0500 2009
Tue Nov 10 06:00:00 -0500 2009
Tue Nov 10 06:00:00 -0500 2009
Tue Nov 10 06:00:00 -0500 2009
Tue Nov 10 06:00:00 -0500 2009
Tue Nov 10 06:00:00 -0500 2009
Tue Nov 10 06:00:00 -0500 2009
Tue Nov 10 06:00:00 -0500 2009
Tue Nov 17 06:00:00 -0500 2009
Tue Nov 17 06:00:00 -0500 2009
Tue Nov 17 06:00:00 -0500 2009
...snip...

What’s the Fix?

None as far as I know. I’m sure one could dig into the internals of the library and figure out how to deal with this problem, but I’m not up for that right now.

One thing I’ve always heard about learning a new language is that you can’t consider yourself fully fluent until you can tell a joke. Telling a joke requires the understanding of homonyms and how words flow together. “So a guy walks into a bar…” sounds different than “A man enters beverage store…”.

When dealing with clients, it’s easy to joke about your client, but don’t miss an opportunity to prevent future missteps once you know more about them.

How often do you get off the phone with a client and immediately have a funny remark about some critique or issue they have mentioned. Typically this manifests itself when the client brings up an issue, that while important to them, is considered trivial by the developer.

Can you believe this guy!? I built out this whiz-bang feature and he’s just complaining about the font being too small!

The key part of this interaction is that you’ve now got a little insight into what makes this particular client tick. When you demo the next few features and he is unimpressed by the functionality, but comments on the spacing of form elements you should skip the joke and make a mental note for the future.

Once you can tell a joke about your client, before the interaction, and you have an “I told ya so!” moment with yourself or your pair afterwards, you know you’ve made it to the next level of understanding that client. Now, the next time you deploy a feature, discuss requirements or ask a question you can preempt the inevitable by accounting for those quirks and personal preferences of your client.

Instead of making a joke, make them happy, then you can both smile. ;)