Similarly to mocking, which is a technique that enables a style of testing different to the classical, we could begin to talk of a testing style made possible by the use of macros and call it structure-based testing.

I’ve already blogged about Shoulda and its macros, but given the recent news about the Shoulda - RSpec love affair and the remarkable increasing popularity of Shoulda-like macros, I reckon it’s time to revisit the subject.

I find appropriate to call this style of testing structure-based because it doesn’t involve the interactions nor the changes of the state, but the actual structure of the object under test. In that sense, structure means the opposite of behaviour. These tests don’t say anything about the behaviour of the object, just what it should look like.

describe User do                             # class User < ActiveRecord::Base
  it { should belong_to(:account) }          #   belongs_to :account          
  it { should have_many(:posts) }            #   have_many :posts             
  it { should validate_presence_of(:email) } #   validate_presence_of :email  
end                                          # end                            

The thing is, I think there’s hardly any value in this kind of testing. Obviously these tests are easy to write and quick to run. The cost of writing them is low so the benefit could also be low and still be a good investment. However, there is usually an almost exact correspondence between the spec and the implementation. It would be possible to automatically generate one from the other. I can see some remote value in thinking twice about the structure of the object, but that won’t give me any clue about whether that structure will provide the behaviour I need to implement the current functionality. I’m just writing the same thing twice, but with different languages.

Unit-testing Rails applications is not straightforward. Dan North justifies this by blaming the framework itself and the coupled nature of its components. Shoulda macros gives you a shortcut, they “make tests easy on the fingers and eyes”, that’s the framework’s motto. And they certainly provide what they promise: instant gratification, effortless testing with high code coverage and relatively fast execution time you can perform “all the fucking time”, like the cool kids do. The question is, does it provide any real value to the development process?

The answer depends, of course. It depends on each developer, team and context. My current approach of Rails testing is pretty different. But that might very well change in the future. The success probably lies in continuously challenging the why and the how of the way you do things, keeping in mind that there is no silver bullet and that, ultimately, it’s all about effectively building software that provides value to someone.

1. We perform testing on two different levels: Acceptance and Unit.
2. For Acceptance Testing we use Cucumber (formerly Story Runner). Cucumber automatically fills the gap between a user story written in plain English and an executable test.
3. For Unit Testing we use RSpec. We could also use Shoulda, Context+Matchy or even Test:Unit+Mocka. Any of these would do. We believe it’s a matter of taste, essentially you can do the same with each of them.
4. We find Acceptance Testing excellent as a verification tool, but not as good as a design/development tool.

5. We therefore need Unit Testing as a development/design tool to enable us to be more productive and to make our code more maintainable.

6. We follow an outside-in approach driven by the user story. We try to write the minimal code necessary to make the next step pass. From the view to the model, outside-in, adding the corresponding specs on the way.
7. A comprehensive set of acceptance tests is essential to the way we do specs. If we didn’t have it, we would unit-test in a completely different way.
8. We understand that “Mocks aren’t stubs” concludes that there’s no winner between classicists and mockists. It’s all about trade-offs, didn’t you know?
9. We only spec the interesting behaviour of the object under test, the behaviour that other parts of the system rely on.
10. We don’t usually spec views. We think it is unnecessary with acceptance testing.
11. We spec controllers following an interaction-based approach using mocks. Controllers have no state, their interesting behaviour is determined by the way they interact with other objects (models and views). That’s why we think interaction-based testing is the most appropriate for controllers.
12. We spec models following a state-based approach. And Rails fixtures are evil. Instead we use any implementation of the Factory pattern (fixture-replacement, FactoryGirl, machinist…)
13. We don’t use macros in our specs. Macros encourage a style of testing I call structure-based (vs. state-based or interaction-based), which, in my opinion, gives almost no value. Using them feels like writing the same thing twice. Perhaps because that’s exactly what you’re doing.
14. BDD works better when accompanied by other agile practices: pair-programming, refactoring, interface-first, short iterations, etc.

15. We are opinionated, like our favourite web framework.

You can find the slides of the presentation here (in English) and the video (in Spanish) here.

I’m working on a series of posts to elaborate on each of these points. Stay tuned!