code.isdangero.us

Announcing rspec-fire-roles

2013-02-25T19:08:31+00:00

TL;DR: The rspec-fire-roles gem can allow for more flexible design and better, more useful failures in your specs. For use with rspec-fire.

After having read Growing Object-Oriented Software Guided by Tests and Practical Object-Oriented Design in Ruby, the benefits of the so-called “London school” or “mockist style” of TDD finally crystalized in my mind. I began reaping the benefits of interface discovery through mocks to design more flexible software organically.

If you are having trouble understanding the advantages of mockist testing, I highly recommend the first book linked above (commonly known as “GOOS”). It may be written with Java and therefore a bit of a pain to read for those of us spoiled by Ruby’s clean syntax, but it’s more than worth the effort. It dispelled misgivings and misunderstandings about mockist testing that I’d harbored for years, and as a result I have a significantly leveled up as an engineer. (Disclosure: Them be affiliate links.)

The problem with mocks in Ruby

Due to Ruby’s dynamic, duck-typed nature, tests focused on interaction using mocks do have one big weakness. Since mock objects are standing in for real collaborator objects, and since Ruby doesn’t have interfaces defined at the language level, your mock-based tests are liable to report false positives if the real collaborator objects in production aren’t compatible with the interfaces specified by the mocks. This isn’t a problem in Java, of course, because it’s a typed language with language-level interfaces, which allows the compiler to catch such situations. Here’s an example of the problem:

One approach to this issue is to write additional integration tests to ensure the objects plug into each other correctly, but the combinatorial explosion inherent in integration testing makes this a weak solution at best.

Enter rspec-fire

An alternative approach is offered by the rspec-fire gem. This gem allows the name of a real class to be specified when creating a mock object. If a test is run in isolation (awesome for writing fast specs), the mock behaves just like any other mock. However, if the test is run as part of the entire suite, thus with the named class having been loaded, the tests will fail if expectations on the mock do not have a matching implementation defined on the given class.

This is a great approach which largely solves the problem. However, part of the problem still remains. When using mocks for test-driven design, you want to mock roles, rather than concrete objects. Thinking in terms of roles results in more flexible software. A given concrete class might play multiple roles. Furthermore, a given role might be played by more than one class. Ruby’s duck-typed nature makes it an ideal language for this type of design thinking, allowing what is probably the most flexible possible application of polymorphism. POODR (the second book linked above) goes into further detail on this topic.

This approach is already halfway doable in rspec-fire. Define a class named after your role with an empty implementation of that role’s interface, require it in your specs (which should even be fast enough in isolation), then pass in the name of that role for your rspec-fire mocks rather than the name of a concrete class. Like this:

This works well enough, but there still isn’t an easy way to ensure that every concrete class playing the Notifier role is implementing the correct interface. If the role changes, the BatchSender spec will fail, but the specs for EmailNotifier, SmsNotifier, and any other concrete class which is supposed to be implementing the role will continue to report false positives.

Enter rspec-fire-roles

The new rspec-fire-roles gem does something really simple: it adds an implements_role macro for use with RSpec. That, and it includes a README which recommends using rspec-fire with roles rather than concrete classes, as explained above. Here’s how you would use it, continued from the above example:

There. We’ve closed the loop. If the Notifier role changes, whether it be from the standpoint of the role itself, any classes implementing the role, or any classes dependent upon the role, the specs will now fail in all of the right places.

See the rspec-fire-roles gem on GitHub for installation, usage, and more waxing on the topic of mocking roles. Feedback and discussion is welcome. See “Future improvements” for ideas on where to start if you’d like to contribute. It is my hope that this gem is useful, and results in faster, better tests, higher quality software, and happier engineers.

Optimizing Rails and Unicorn on EC2

2012-07-18T03:47:57+00:00

In a recent project, we had the challenge of ensuring the scalability of our app up-front, as the client would be sending tons of people to the app after launch via their Facebook Page fans and their marketing mailing list. We needed to make sure that the app would withstand the load and scale out effectively if needed. We extensively tested a number of configurations of our fairly standard Nginx + Unicorn setup. I thought I would share our optimizations and findings, including the type of EC2 instance we found to be most cost-effective for our app servers.

Optimize for request length

This seems almost too obvious to point out, but the level of concurrency your app can handle is largely limited by how long it takes your requests to complete on average, so much of the work you put into scalability should be focused on speeding up any slow requests. If your app takes 1000ms to serve up an average request, you’re not going to gain much by horizontally scaling out multiple instances of your app. A few milliseconds’ difference in request time isn’t much to worry about in most cases, but a good rule of thumb is to try and keep the total request time under 250ms. Keeping it under 100ms is even better.

There are two great tools out there for helping to diagnose slow requests. New Relic is the heavy-hitter in this arena, however Scout just recently began offering a more lightweight version of what New Relic does, and at a more appealing price point. And, as of today, the scout_rails gem has a beta release which supports Unicorn; I’ve been running it in production and it’s working splendidly. It doesn’t offer the same level of detail yet that New Relic offers, but they seem to be iterating quickly on it. I recommend checking out both tools to see what fits your needs and budget.

Background as much as you can

Any time your request does any significant work for which it does not require the result to proceed, background it, especially if that work could take a variable amount of time. This could be something like mailers, which are safe to fire and forget. In our case, we were creating analytics summaries at the end of each request, which ended up taking far more time than we anticipated. We moved that work into a Sidekiq job, and our average request time fell dramatically.

Cache as much as you can

HTTP caching is the holy grail, with page caching coming in second, because both of these methods completely avert the need for requests to even hit your app. However, if your app is highly dynamic, these methods won’t help you much unless you use Ajax to pull in dynamic content after the initial page load; such an approach may or may not be worth the extra effort, depending on your app. They also make it difficult or impossible to restrict access to parts of your app based on the user’s session.

Action caching is more likely what you’ll be able to use when you can, and when that’s not granular enough, use fragment caching. Use self-expiring keys everywhere so you don’t have to sweep caches manually or rely on cheap time-based expiration. Use Russian doll caching with your fragments where appropriate to keep everything nice and tight.

Don’t take this lightly! Even for simple views, ERB can take a surprising amount of time to render. I had some requests where view rendering was actually the bottleneck, with ActiveRecord being a distant second. If you can cache it, do it.

37signals does a great job of explaining the techniques of Russian doll caching and auto-expiring keys. And this article has a thorough overview of all the different types of caching available out of the box in Rails, which I highly recommend reading if you’ve never used Rails caching or need a refresher on it.

Make Nginx take advantage of the asset pipeline

First off, make sure that Nginx, not Unicorn, is actually serving up your static files. We realized we forgot this when we saw requests for assets in our logs. Fixing this drastically increased our app’s throughput. Second off, make sure Nginx is serving the pre-gzipped versions of your assets generated when you precompile them. This section of the Rails Guide to the Asset Pipeline explains precompiling assets and properly configuring Nginx to serve them.

Note that when you do this, you will need to make sure that you’re using Rails helpers wherever your assets are referenced, such as linking to your CSS and JavaScript files or generating image tags. Also, in your SASS, url("style/active.png") should be rewritten as image-url("style/active.png").

This step alone made a great difference in how quickly our app served requests, and therefore to how many users it would be able to handle at once.

Configure Unicorn for one worker per core

Having deployed Mongrel before, the natural conclusion when configuring the number of Unicorn workers is to take advantage of your RAM and have as many as possible. This is not the case with Unicorn, as we discovered. In our testing, we found that just one Unicorn worker per core far outperformed even a single additional worker. We were really surprised by the difference. You may want to test this yourself with your own configuration, of course, as the Unicorn tuning documentation does state that additional workers may help in certain cases, but we found that exactly one worker per core was optimum for our application.

Do your own load testing

This brings me to one of the last tips, which is do your own load testing! One of the beautiful things about on-demand EC2 instances is that it’s cheap and easy to duplicate your production environment and try different types of instances and configurations and then discard them when you’re done. We took full advantage of this fact. Indeed, we ended up spinning up and tearing down so many instances that we took a little extra time to write a script to help automate this process using the AWS gem, which will of course pay dividends in future productivity as well.

For the actual load testing, there are many routes to go, from as simple as the Apache benchmark tool to more elaborate, distributed setups. For our fairly simple app, we found Blitz.io to be very fast and easy to get started with, and just better at simulating actual load than we would be able to from our laptops. It’s also really affordable, and in our case, most of the load testing we did was actually free. I recommend giving it a try! Disclaimer: If you sign up for Blitz.io after clicking that link, our limits on free usage will be bumped up slightly. :)

Use c1.medium instances

Okay, this isn’t exactly divine law, but our testing found that the c1.medium instance type gave us the best bang-for-buck as far as pushing the limits of concurrent users our app servers could handle. Rails and Unicorn, it seems, are CPU-hungry, and the more you can give them, the better. We also tested the other high-CPU instance type, the c1.xlarge, and while fewer instances were needed to achieve the same level of concurrency, we found that each concurrent user was more expensive than with the c1.medium instances.

Ultimately, we did reach a point where adding new app servers gave us fast-diminishing returns. In fact, we found that having six c1.xlarge instances, averaging at about 530 concurrent users in Blitz.io before the app began serving error 502, performed worse than just three of the same instance type, which would start crapping out at 562 concurrent users. This would seem to indicate that, at this point, our app servers were no longer the bottleneck, but that probably our database (MySQL on an m1.large instance) was. However, our optimization had gotten our app to the point where we were most likely exceeding the speed we actually needed, so we were happy.

Conclusion

Hopefully this helps someone else out there who might be thinking about deploying a similar configuration to EC2, or who might have an existing deployment but is mystified as to how to get more performance out of it. If you have any other tips I may have missed, please feel free to add them here so I can add them to our app as well! :)

DRY specs with ProgressiveRequirements

2012-04-13T03:04:17+00:00

Anyone who has spent significant time crafting automated tests with RSpec or Shoulda knows context nesting hell. This seems to be a common idiom with specs which are building up toward a successful scenario while testing failing scenarios along the way. This makes sense, because you’re testing different layers of requirements which seems perfectly suited to nesting contexts. That is, until you need to read the resulting spec, or make modifications to it down the road.

Here’s what I’m talking about:

Note that I’m using a regular method for request_params rather than let or let!. This is necessary so I can use super and keep the requirements DRY. You can’t access the parent context with let (at least not yet).

Now, this isn’t half bad. But imagine if you had more than just three requirements to satisfy, and each one had its own consequence for not being met. Now imagine you wanted to also share these requirements so they could be reused, mixed and matched for different controller actions. Now you have to extract them into shared examples or shared contexts. Now you’re in nesting hell.

After lots of head-banging (and not in the musical sense), I came up with a different approach and a helper to make it easily repeatable. It lets me rewrite the above example like this:

What sorcery is this?!

First we declare our requirements and how they are added to the spec. Then, in our spec, we include the requirements one at a time, including what should happen if the requirement is not met. Finally, we spec what should happen with all requirements met.

Note that the order matters. In the on_failure block for requirement_3, the changes made by requirement_1 and requirement_2 have been added, but not the changes for requirement_3. And after all the include_progressive_requirement declarations have been made, and all the requirements are met, so you can define what should happen at that point.

This also makes them reusable. Because each requirement makes an atomic change to your scenario, you can mix and match them or use them in a different order if necessary for other specs. In my opinion, the resulting spec is also easier to read since the nesting is simply not an issue. I would rather write and modify the spec using this approach than the highly-nested spec.

How does it work?

Just include this module:

Note that you can also include on_failure in the requirement definitions if you’d like default failure examples every time the requirement is included. If you would like on_success, it would be easy enough to add as well.

Possible improvements would be to allow add_requirement to do more than just override existing methods, but add before blocks as well to do things like stub out API requests with required responses. This would also require defining a remove_requirement block to undo such a change in the failure context. I’m going to continue using this and see how useful such additions might be.

In the meantime, any input on this approach is greatly appreciated! I know I’m not the only one on a quest for DRYer, more maintainable specs, and this is just one approach I came up with which I think is unique. Suggestions on alternative approaches are greatly appreciated!

The Command Pattern: A Lightweight Alternative to DCI

2012-02-09T02:16:21+00:00

Recently the Rails community has seen a flurry of activity regarding a new approach to encapsulating business logic in Rails called Data, Context, Interaction, or, DCI. And rightfully so! The old mantra of “fat models, skinny controllers” has cracked with wisdom as models in larger and more complicated applications have become not just fat but in fact rather obese. Furthermore, the question of which model some piece of business logic should live in has always been vague regarding operations involving multiple models of different kinds.

It is a problem. The results have been sub-optimal design, testability, and readability, coupled with huge model files bloating into thousands of lines of code. The DCI approach has been explored at length as a possible solution. Mike Pack wrote a great article both explaining the benefits of DCI and an example implementation.

The DCI Pattern: Pros and Cons

There are some clear benefits of the DCI approach, especially compared to the old way of just shoving business logic into the nearest relevant model:

Skinny models. DCI effectively cleans up your models, and should leave them with nothing but declarative associations, validations, and callbacks, plus any convenient accessor methods and query scopes. Models are left dealing strictly with the access and persistence of your data. In this way, they are dumb, knowing nothing about the business logic beyond how to find, validate, and persist the data for which they are responsible.
Explicit definition of user roles. In the Agile development methodology, user stories are defined as being the goal a user who represents a given “role”, such as a “Customer”, “Client”, or “Administrator”. In DCI, abilities tied to these roles are defined explicitly in their own Role modules which extend the actor instances at runtime.
Clarity. As mentioned before, when working with the fat models approach, it’s often ambiguous as to which model a given set of business logic should live in, especially when that business logic involves manipulating a number of different kinds of models. With DCI, each business goal lives in its own Context class, regardless of the types of models it requires, so there is no confusion.
Testability. With the clean separation of logic, it’s very easy to test each user story in isolation.

Sounds great, right? It turns out that DCI is just the kind of scalpel Rails needs to divvy up responsibilities around complicated business logic, and its nearly lock-step fit with Agile user stories is a great extra feature. That said, here are some of the drawbacks that I can see:

Performance. One of the main bogeymen regarding DCI right now is performance. The “correct” approach to DCI is to extend the actors with their appropriate Roles at runtime, which can be a significant performance hit if done enough times. In larger applications with a high request volume, this can be significant. Mike Pack has written another article addressing different approaches to injecting Roles into actors, each with its own trade-offs of performance and purity. It’s definitely worth a look.
Extra classes/modules and boilerplate. Each Role requires its own Module, and each Context must begin by extending the given actors with their appropriate Roles. It’s just additional overhead during development.
More moving parts. Each Context requires an intimate knowledge of the actions available to the Roles involved.

I still think DCI is an excellent solution, but it occurs to me that it might be a little heavyweight for some applications. That’s why I’m proposing a lightweight alternative which has been a steadfast design pattern for nearly two decades.

The Command Pattern: Pros and Cons

The Command pattern has long been the go-to pattern for encapsulating undoable/redoable actions in GUI applications. I used it in the development of my game, Elite Command, to great effect. My original intent was to add multiple undo to the game, but then it had the great side-benefit of removing a lot of game logic from my models and separating possible player commands into their own easily-testable classes. The result: skinny controllers, skinny models, and relatively skinny Command classes each concerned with an individual feature of the game.

Here are some of the advantages to the Command approach, many of which are the same as for DCI:

Skinny models. Like DCI, the Command pattern encapsulates individual business goals into their own classes, leaving models concerned purely with access, validation, and persistence.
Clarity. Like with DCI, there is no question as to where the logic for a given business role should go: it goes into its own Command class.
Testability. Like with DCI, since each business goal lives in its own class, it is supremely easy to test each scenario in isolation.
Peformance. Unlike DCI, the Command pattern requires no extension of objects at runtime. Commands are merely Plain Old Ruby Objects hiding a business goal behind a simple interface. Indeed, the Command pattern can in many ways be regarded as DCI without the Roles.
Fewer classes, less boilerplate. Without Roles or runtime extension of objects, one only needs to look in one place for a complete step-by-step view of a given feature’s functionality.
Undo and redo support. The Command pattern was originally designed with undo support in mind, so it’s a natural fit if you plan to add undo or redo to your Rails application. Of course, it’s basically just DCI without Roles, so I don’t see why DCI couldn’t also do this fairly easily.
A natural fit with the Composite pattern. The Composite pattern allows a tree of objects to be treated as a single object. An advanced implementation of the Command pattern might include a CompositeCommand class which allows multiple Commands to be executed in sequence. This can be useful for things like minimizing Ajax requests by bundling multiple Commands into a single CompositeCommand, a technique I used to great effect in Elite Command. DCI may be capable of this as well, although I’ve never seen an implementation.

I like the Command pattern for its simplicity. Of course, there are always trade-offs. Here are some of the cons to the Command pattern versus DCI:

No explicit Roles. You don’t get the one-to-one mapping of Agile user roles to Role modules that you do with DCI. This can be a nice thing to have, depending on the relative complexity of the application
Not necessarily as DRY. I can imagine some circumstances where Roles might be a good place for code which is reused between Commands.
Coupling between Commands and model interfaces. Where DCI has coupling between Roles and models, the Command classes have coupling between Commands and models, with no intermediary. This might not be a big deal, but it’s something to keep in mind.

The real disadvantage to the Command pattern versus DCI is that Roles are not present. This results in fewer classes and less boilerplate, but as a trade-off you may need to place some logic which would go into Roles into either a Command superclass or into one of the models themselves in order to keep the code DRY. In Elite Command, I opted to include shared logic as modules into the Command class, which has generally worked fine.

The main advantages over DCI are the performance gains from not having to extend anything at runtime and the fewer classes and lines of code required to support the pattern.

An example of the Command pattern in action

Here is a simple implementation of adding an item to a shopping cart using the Command pattern. Elite Command actually uses a much more sophisticated, fully-featured implementation which I may cover in a future article, but for now here is a quick demonstration.

First, our simple models:

And now, the Command superclass:

And finally, a spec and implementation for adding an item to a cart:

Simple, no? Of course, because it’s an exceedingly simple example. However, Commands really shine when they encapsulate more complex algorithms involving interactions between multiple models. And with a dash of persistence and the addition of #unexecute! methods, we have really easy undo support ready to go!

As mentioned above, there are more sophisticated features which can be attained with the Command pattern and which proved to be a great boon for Elite Command. If there’s enough interest, I may write on this further in the future.

Conclusion

As always, design decisions come with trade-offs. I am not advocating the use of the Command pattern instead of DCI. Rather, I am suggesting an alternative which shares many of DCI’s benefits but with a slightly different set of drawbacks. Before going with one or the other, ask yourself whether the explicitly defined Roles are worth it to your application, or whether the lighter conceptual overhead and smaller performance footprint of the Command pattern are of greater concern.

Only tools use too many tools

2011-06-13T23:19:08+00:00

One thing I’ve found that most engineers dislike is the overuse of tools. I’m not talking about build tools or other command line tools, which automate repetitive tasks and make our lives easier. I’m talking about the tools that management wants everyone to use, whether it be for bug tracking, feature reviews, project management, or any other work that is peripheral to the actual task of writing software.

These tools aren’t inherently bad. They can be of great assistance to the process of deploying quality, production-ready code. But I’ve encountered the situation more than a few times where these tools have done more to get in the way than they have to actually help. Usually it’s either due to not picking a tool and sticking with it, or to using tools where they aren’t necessary.

Exhibit A: Too many tools

Here’s one scenario from real life. One company I worked at used five different tools for tracking progress. Five. Two is one too many, and they had five. Trac, Pivotal Tracker, Basecamp, and a couple others whose names I don’t even remember, all played into some sort of chaotic dance to which nobody in the company knew the steps. Every project had its own tool for tracking bugs and progress. Nobody knew which tool to go to for updates. It was a mess.

They eventually moved to a convoluted, over-engineered JIRA setup, but I won’t get into that.

Exhibit B: Redundant, low-quality tools

Here’s another scenario. A company is using GitHub to manage its code repositories. Great! GitHub has a lot built-in, much of which can be put to good use by companies of many sizes. But this same company is also using a completely separate tool, Review Board, to do code reviews. Using this tool involves the complicated process of running a special script for rebasing and diffing your feature branch, which can break for many reasons and leave your local repo in a weird state. Even if you do succeed in generating the correct diff, you have to then manually upload it to the Review Board software.

Why go through that trouble? GitHub has Pull Requests built-in and has for a long time. They provide a diff of your branch as well as a discussion thread for that diff. You can track changes as you go, and when it’s ready, just click a button to ship it. What more do you need?

Exhibit C: The right approach

One last thing. Many of these tools are more to help managers do their jobs than engineers themselves. To that end, I think it’s a good policy to limit engineers’ exposure to such tools as much as possible. Let them spend their time writing software.

Another company I worked with had a great approach; engineers were not expected to update Trac at all. Each team would have a sprint meeting with the relevant managers at the beginning of each week, where each ticket would be addressed via projector and the manager running the meeting would take care of interfacing with Trac. It was a marvelous approach. We engineers could focus on writing code and the managers could focus on tracking progress. The way it’s supposed to be.

Conclusion? Choose your tools wisely. Don’t use tools redundantly.

Basic funnel tracking in Rails

2011-02-13T00:27:05+00:00

This is the first in a series of articles I intend to publish about the technology, philosophy, and process behind my latest Rails project, Elite Command. Elite Command is a turn-based, multiplayer strategy game, and a lot of interesting approaches were taken in developing for its unique requirements.

Before I get too into the grittier aspects of the game, I’m going to cover something which applies to any web application which strives for commercial success: analytics.

Having designed and maintained various analytics platforms over the last three years, I’ve learned a thing or two about best practices and capable architectures for supporting the requirements of a robust analytics infrastructure. I’ve also learned a lot about the types of metrics which are most useful in a business sense. With this knowledge, adding analytics to Elite Command was a matter of choosing the 20% effort which would yield 80% of the useful results. That’s why the first analytics tool I baked in was funnel tracking.

Note that my code is using MongoDB with Mongoid for the ORM, but that this is all just as applicable to whatever data store you may be using.

What is this funnel of which you speak?

Funnels are incredibly useful for exactly one thing: tracking conversion rates of users through a flow. That’s why it’s called a funnel. In any given user flow, there is a chance for users to drop off, leaving fewer and fewer remaining users who actually reach the final step. A funnel allows you to pinpoint exactly where you’re losing users, and thus gives you some evidence on which to base targeted improvements with the goal of converting more users each step of the way.

This is extremely valuable when trying to build a user base. The first funnel you would want to measure is naturally the experience for brand-new users. From the moment they arrive at your landing page from a given source (such as an ad) to reaching the sign up page, to signing up, to engaging with your application, you want to know where users are giving up and leaving, and you need hard data to support your theories and measure improvement. This is especially important when users are coming from ads, as you want to make sure that you are maximizing the cost-effectiveness of paying for clicks or impressions.

The source and the cookie

The first thing you want is some kind of trigger for slapping the user with a unique identifier when they come from a trackable source. For this purpose, Elite Command looks for a query string such as ?src=cid_0_2 (which is indeed included in the link in this article) and saves that, along with a unique tracking ID, to the user’s cookies. The src string can be anything, but I recommend coming up with a standard format which makes sense for your needs. Mine is “[source name] _ [creative number] _ [release number]”, allowing me to measure the effectiveness of ad channels, individual ads, and the actual release of Elite Command. Here’s the code for creating these cookies:

The tid is the unique tracking ID, which comes from a class method on User. Here’s the code for that:

The UUID class comes from the UUID gem and generates a Universally-Unique Identifier, simply a string which is by all practical means completely unique within the scope of the application.

Now we have a cookie which uniquely identifies the user and tells us the source from which they came.

The User model

Cookies are great, but if a user signs up, we may want to know throughout the application’s lifetime which source they came from. We can use this to make funnels which stretch across multiple uses of the application. For example, we may want to create funnels which track whether users respond to emails sent by the app. To do this, it just makes sense to save our tid and src to the user record itself. This is pretty simple. First, just add the fields to your model (here using Mongoid):

Notice that we have the ensure_tid callback. This attaches a unique identifier to every user, even if they did not get tagged through one of our sources. This simply means we can track them within the same system.

Now every new User has a tid, and they also have a src if they came from one of our predefined sources. Now it’s time to put this all to use and start recording some data.

The UserAction model

Now we need a way to record when an action is taken, along with the tid and src, if available, of that user. Behold, the UserAction model:

That’s all there is to it! Now we can call UserAction#record whenever we want. You can pass in both a User instance (which can be nil) and a cookies object, and #record will do the right thing with it.

Funnels!

Now we can create a Funnel model. This model consists of a series of ordered steps (or UserAction#name strings) and a name. We also need a way to calculate the numbers of unique tids recorded at each step. Without further ado, here it is:

There you have it! I won’t go into details on creating the admin UI for creating and viewing these funnels, but it should be pretty straightforward. Funnel#step_results(src) returns an array of pairs, each pair consisting of the name of the step and the number of unique tids remaining at that step. There are many ways to display this data. I settled for a simple column of numbers and percentages with CSS-generated bars representing the portion of users remaining at each step. Whatever lets you see the data.

What next?

Using this data, I’ve made several ad runs using Facebook Ads and tracked the effectiveness of the new user experience. The goal is to get users not only to sign up but to become active users of the game. With each ad run, I can see exactly where the flow could use the most improvement. This is where I get creative, imagining what might entice the users to keep going forward. I implement and launch the improvements, do another ad run, and then look at the funnel numbers for that source. From this, I can determine whether the improvements worked or not and continue iterating toward an optimal new user experience.

There is certainly room for improvement. For one thing, it would be nice to track source by HTTP referrer for untracked users coming from an unknown source. It would also be nice to calculate the funnel in a more strict manner, ensuring not only unique tids at each step, but also enforcing the order of the steps. And, as I’m sure goes without saying, there is plenty of room to improve on performance; the funnel numbers should be calculated using map-reduce, and there are some missing indices as well. But this works for the immediate need, and I’ve been able to use this system to improve retention of new users considerably.

Let me know what you think and if you have any ideas for improving the system. Also, be sure to give Elite Command a whirl and tell me what you think, and don’t forget to invite your friends! I hope this article proves useful to someone else’s project.

The virtues of flying solo

2011-01-27T14:04:27+00:00

In the last month, I’ve had ample time for the first time in a long time to really dive into a solo programming project. It was something I had actually been working on for longer than that, but not until this last month have I had the rare chance to work completely uninterrupted on my own. The result is a product launch—that’s right, this thing is shipping, next week! When that happens, I will write a more detailed post about some of the approaches, fairly unique to the Rails world, which I took to a few different problems.

But first, I wanted to write a post reflecting on the merits of working alone as a programmer. There are many articles which talk about the different challenges and advantages of working in teams, but rarely on programming solo. This makes sense, to an extent, as most of us have Actual Paying Jobs™ and work with teams more than we work alone, and there is also all the ongoing buzz about the benefits of pair programming over the last several years. At the same time, though, it’s surprising, since most of the best programmers I’ve met on various teams got their start doing it on their own, as a hobby, as an amateur, a lover of the craft, and only joined companies when they realized they had bills stacking up.

If you’re one of those programmers, think back. Remember that? Remember the first time you got your compiler working? Remember fixing your first bug? Remember the first time you programmed something, not because you were trying to learn a skill or complete a job requirement, but because you were curious to see if it could be done? Let’s take a step back to that time for a moment.

It’s not (that) dangerous!

When you’ve spent years working on teams where practices such as regular code reviews and pair programming are enforced as part of the development process, it can be easy to get the general impression that working on something by yourself, in a vacuum, as they say, is a Bad Idea®. I want to dispel that right now. It’s actually a Really, Really Good Idea®!

Don’t get me wrong. Code reviews and pair programming are great ideas for teams, if they’re done in the right way. The wrong way is to have the attitude that it’s done to prevent incompetent code or designs from getting into the ever-growing repository. Well, okay, that is true to a small extent. But the real reason code reviews and pair programming are a good idea is not because we don’t trust our peers; rather, it’s because working on a team introduces a new challenge which solo programmers don’t face: knowledge sharing. It prevents someone from becoming the only one who knows about parts of the system, which makes them a liability if they get hit by a bus, decide to move on, or otherwise leave the project.

Solo programmers don’t have this problem! This makes irrelevant the bulk of what code reviews and pairing are meant to address. Sure, if something plays out and the solo programmer goes writing code that ends up becoming popular as open source, or ends up building a business and having to hire more programmers, then someone else will need to learn the code. However, at this point, assuming there are good tests in place, it should be fairly simple. Indeed, I’ve found that working on the guts of open source tools which originated from a single programmer is consistently easier than with code written by an organization. Perhaps the code lacks some optimizations that the individual did not know about, but at least there’s a greater likelihood of consistency in things like naming and stylistic conventions, as well as a more cohesive design altogether. (Naturally, this assumes that the individual is a programmer of reasonable talent.)

The point is, individuals can and do build great, reliable software. In fact, free of the risks and pitfalls inherent in working on a team, magical things can happen.

No interruptions, no compromises

Another one of the challenges brought in by working with a team is the constant need to stay in sync. This brings in things like status stand-up meetings, sprint meetings, design/planning meetings, and the like. Throw in code reviews, pair programming, the usual interruptions from support, QA, company meetings, and lunch (because we gotta eat), and one might rarely get the chance to just sit and do some quality, solo programming. While many of these things have their place, I know I’m not alone as a programmer when I say that one of my favorite things about programming is being able to put some tunes on in the earbuds, shut out the world, get into the flow, and slam out some fine, fine functionality. Such is a luxury, and often one nobody can afford, when working on a team, especially larger teams.

But for a team to work, everyone needs to be on the same page. The solo programmer is already on the same page, assuming the lack of conditions such as schizophrenia. The solo programmer doesn’t need planning meetings or design meetings. The solo programmer devises a solution and then makes it happen. The solo programmer doesn’t need complicated development processes checking and double-checking his every move, nor similarly-complicated bug tracking software. The solo programmer has his vision on scraps of paper or a simple to-do list app, and when the solo programmer wants to check his progress, he simply looks at what he’s accomplished.

Maybe the solo programmer doesn’t come up with the most optimal possible solution. But given what I’ve seen and the kinds of egos harbored by the most talented programmers, neither does a planning meeting or analysis by paralysis (reversal intended). Even at its best, a planning meeting certainly doesn’t get any actual work done. At least the solo programmer, proceeding audaciously with no more advice than could be gleaned from a Google search, a reference manual, and past experience, gets things done.

The glorious return of the individual hacker

We as programmers can sometimes forget the power that we wield in today’s age. People of all stripes in the industrialized world are reliant upon the technology which we know how to shape and form and apply at a whim. Most people interact with software every single day, often without even being aware of it. The internet itself is ever expanding its reach into the homes of more and more people, all over the world. For those of us with the knowhow, computing technology is in a unique position to spread innovation and to disrupt industries. And of those among us with the knowhow, a single visionary, working independently and with determination, can accomplish a lot, including much that a team cannot.

There is no reason that any programmer of reasonable talent, anywhere, should feel as though they require the stability of someone else’s vision and guiding hand. If we so choose, we individuals can be the sole visionaries and guiding hands of our own labors. I can cite many cases of this happening, but I should not need to; if you’re working in the tech industry, you are likely mere degrees away from someone who made exactly that choice, and, indeed, probably working for them.

I can’t speak for everyone, but I can say conclusively for myself that the number of times I have felt the fleeting moment of utter joy at what I’d done while working for someone else absolutely pales in comparison the the number of times I’ve experienced that feeling working on my own projects, even those projects which never panned out into anything greater than “for my eyes only”. I can say this with confidence, even while I have had the opportunity to build some pretty wicked shit, if I do say so myself, while working for the man. Programming solo on a project I wanted to do just to do it always feels infinitely better than working on someone else’s project for a paycheck, even if they have a pretty cool project. Why? Because your own project is your own project. You can only fake that level of enthusiasm for so long.

Make it happen

I will end this post with a call to action, one which echos what I heard Hampton Caitlin preach at a Ruby conference, reminiscing about when he built Haml.

That side-project you’ve been mulling over in your head? The one you know you could build? The one you could build so well, so elegantly, you can almost see it unfold in your mind’s eye? The one that you would love to at least start on, but you feel so tired after a long day at work, and those days at work just don’t seem to be getting any shorter, even as the rate of pay remains the same?

Build it. Work late. Spend the energy, even if it takes a toll on you. Just do it. Buy yourself a sixer of your favorite beer, if that’s what gets you motivated, and just code until you can’t anymore. Shut the world out for a while and do what it is you want to do. Be selfish for a little bit. Allow yourself that pleasure, that old, worn, first joy of programming, the one that has been left dusty in your sepia-toned nostalgia in exchange for that regular paycheck.

You’ll be glad that you did. And maybe what you work on blossoms into a thing of its own. Maybe you can quit your job and instead make a living as both visionary and implementor, rather than merely some visionary’s implementor.

Or maybe not. Maybe there wasn’t a market for your idea, or the timing was bad, or it just didn’t gain traction. But you’ll feel better having taken the risk and stuck it out there instead of taking the “easier” mundane route. Who needs security? Fuck security. Security is an illusion, and all the accolades and perks from your bosses can’t give you true freedom. Go get dirty, get bruised, and then get back up and do it again. Throw your creativity to the wind and watch it fly or watch it die, because who cares? Life is short. Make these bets while you can, or die regretting that you didn’t.

SimpleFacebook: A stupid-simple wrapper for the Facebook API

2010-04-02T01:57:38+00:00

I recently encountered a need for a minimal wrapper for Facebook’s API, so I wrote SimpleFacebook. Clocking in at just 56 lines, it’s perfect for interfacing with Facebook via Ruby, and nothing more. This is not intended as a replacement for Facebooker or RFacebook. It doesn’t provide any explicit Rails integration, such as controller filters for gating users. It only does the dirty work of putting together a valid API request and returning the results, deserialized from JSON. Example usage:

What is it useful for? My use case was an asynchronous worker which needed to grab some data from Facebook to be stored in a database somewhere. A Rails process was not necessary to do this, so loading up Facebooker or RFacebook seemed like overkill.

Someone else might find it useful, so I’ve released it under the MIT license. Everything is up on GitHub, which you can download and install as a gem (using rake install). The code is fairly simple, and fleshed out with specs.

SimpleFacebook on GitHub

Bluepill: Low-impact process monitoring

2009-11-06T01:06:08+00:00

At Serious Business, we’ve been using God to monitor long-running background tasks and other daemonized processes. While God provides a nice DSL for configuration, it has issues with memory leaks which forced us to keep a close eye on it (meta-monitoring?) and reboot it periodically via cron.

To remedy the problem, a few of the guys got together over a weekend and wrote Bluepill, a replacement monitoring tool with a DSL inspired by God, but written for low memory consumption. Check out Arya’s blog for a detailed explanation and breakdown of Bluepill’s features. See the graph comparing Bluepill and God memory consumption over time. You’re sold.

Test failure scenarios first

2009-05-19T23:17:24+00:00

Lately in test-driven development, I’ve noticed that I tend to write tests against invalid input before I get to tests against valid input. This is a subtle technique which I think merits some discussion.

One of the big wins with TDD is the way that it forces you to think about the problem in terms of the API; this pushes the engineer to design to an interface, not an implementation, which is one of the major takeaways from the original Gang of Four book on design patterns. And I think the higher-level concern that designing to an interface facilitates is the process of knowing the problem domain, which leads organically into the process of dividing up responsibilities between discrete classes and modules.

In many ways, a problem domain can be defined not only by what it’s supposed to do, but also by its constraints, the cases in which it should fail. Thinking about these cases upfront, testing against invalid or nonsensical inputs, leads to software which is more resilient to misuse and unexpected scenarios.

I find that defining these constraints up front is a great way not only to get the ball rolling. It also ensures that you’re intentionally thinking about constraints as you write your tests. For example, it’s very easy to write simple tests with assert_raise blocks and then have the implementation raise the appropriate exceptions for those scenarios. Just like that, you’ve got passing tests around some basic constraints, such as parameter requirements. Once you have a good set of constraints defined and tests for them are passing, you can start writing tests against the cases which are supposed to work.

If you read my Shoulda tests, you will often see a context start with a bunch of tests that look like should 'fail if X' do ... end. After that, you start to see the tests which test the real meat of the feature.

Besides that it lubricates the initial friction of writing tests, I think this approach leads to solid, secure, and fault-tolerant code. The client code can catch the exceptions as appropriate and react accordingly. Whatever happens, you can rest easy that the API won’t let the caller do things it shouldn’t because you thought of these things up front and wrote tests against those cases first thing. Of course, human nature is that we can always miss cases that we hadn’t considered, but thinking about them from the get-go is much better than trying to rig it in after you already have an otherwise complete implementation.

This is just something which I’ve noticed emerging in my own testing style, which is of course in a continuous state of evolution. Does anybody else have thoughts on testing around constraints? Do you tend to write them first or save them for later? Do you notice any patterns in your testing style which you’d like to share?