Symphonious

The Business of Standards

2012-01-25T21:25:34Z

Recently I’ve been getting spam from the “standards organisation” OASIS inviting a company that I don’t work for1 to join a new standards initiative. There’s no pretense that I’m being invited because of my clearly superior knowledge of the area involved, merely that the company could get great advertising exposure by participating – including being listed on a press release! Naturally we’d need to become OASIS members and pay the appropriate fee, and to be in the press release you’d have to be at minimum a “sponsor level member”.

On the one hand it’s nice that they aren’t pretending to be anything other than a for-profit marketing company, but isn’t it a bit sad that we think standards are just a marketing tool?

1 – yes, they’ve done so much research into how much I have to contribute that they’ve got the company wrong ↩

Development Mode – Concatenating Scripts and CSS

2012-01-11T21:37:30Z

HTML 5 Boilerplate reminded me of an old-school tool which is extremely useful for concatenating JS and CSS files on the fly – server side includes:

<FilesMatch ".*\.combined\.(js|css)$">
  Options +Includes
  SetOutputFilter INCLUDES
</FilesMatch>

Then you have a main scripts.combined.js (or css) which contains:

<!--#include file="libs/backbone-min.js" -->
<!--#include file="libs/underscore-min.js" -->
<!--#include file="libs/jquery-1.7.1.min.js" -->

Plus any other files you need, in the order you specify. This works great for development mode so you can change a file on the fly and just refresh the browser without running any kind of build script. When it comes time to push to production, it’s easy for a build script to process the file ahead of time and be confident that you’ll get exactly the same result.

Cross Pairing

2012-01-03T21:43:11Z

This evening I stumbled across two interesting posts about alternate layouts for pairing rather than sitting side by side. Firstly Tom Dale talking about Tilde’s corner desk pairing setup (and some of the software setup they use) and that was inspired by Josh Susser’s face to face pairing setup at Pivotal Labs.

Both approaches require more floor space which makes them difficult to setup but I would expect the face to face pairing to be a heck of a lot better if done well. I’ve always preferred having separate screens in mirror configuration as well as separate keyboards and mice to allow the developers to sit a little further apart to be comfortable and to be able to look straight ahead at the screen. That said, I quite like having a second monitor for spreading out windows as we have at LMAX so it’s not clear cut which is better.

It’s also interesting to note the popularity of the flat screen iMacs as opposed to Mac Pros or laptops. The former being too expensive for extra power and extensibility that generally isn’t required and the latter often being a bit too individualised to be good as pairing machines. Plus laptops, while amazingly powerful these days still have less bang for the buck and the reduction in performance is just enough to matter for development.

Bottlenecks in Programmer Productivity

2012-01-01T20:40:11Z

Yan Pritzker recently posted “5 ways to get insane productivity boosts for coders” which provides some good tips on how to improve your usage of tools when doing technical work. To summarise:

Never look when you can search
Don’t repeat yourself (by setting up shortcuts and command line aliases)
Learn a scripting language
Learn an editor (and use that editor everywhere)
Learn regular expressions

However, nearly all of these tips really boil down to how to be more productive at writing text and the mechanics of writing code – editing actual source code files, jumping to different places in those files, executing commands more efficiently etc. Are these really the tasks that consume the vast majority of a developers time?

While often it feels like they are, after all we spend all day working with a text editor or the command line, are we really spending all that time struggling to keep up with the stream of code our brains are trying to output? Personally, I spend a lot more time thinking about what the best algorithm is, what direction we should be pushing the design in and how to do that and so on. Rarely do I need to type at full speed for extended periods of time.

Given that, I suspect the real bottlenecks in developer productivity are more along the lines of:

Comprehending an area of code and its design
Identifying, evaluating and selecting potential solutions to a problem (be that choice of algorithm, design choices or choice of classes/libraries etc)
Understanding and sharing a design vision within the team
Understanding and sharing requirements within the team

This isn’t an exhaustive list but it should give an idea for the types of things that really limit programmer productivity. In other words, someone who types with two fingers can be much more productive than someone who knows every shortcut in their editor if they are able to identify a simpler solution to the problem.

For example, recently LMAX needed to make a large and potentially quite risky change to how our system worked. The change affected a core concept in the system and so could have knock-on effects to a wide range of components and be very hard to test effectively. At the start of our two week iteration we already had a potential solution planned out but the developers were uncomfortable with the amount of risk involved with that plan. So we spent a full week discussing options, exploring the code and experimenting, plus talking with business experts to get a better understanding of the domain model. All that discussion led us to find a much simpler solution which, if something was missed, would be sure to cause failures in our existing acceptance tests. Even counting that initial week of discussion, finding the right solution meant we could deliver the change in about half the time we had expected.

So by all means, spend time learning to use your tools better, it avoids a lot of tedious boring work, but if you really want a step change improvement in productivity, work to improve you communication, design and thinking skills.

Searching on Google Maps Does Nothing In Chrome

2012-01-01T19:58:24Z

If you’re experiencing an issue when using Chrome with Google Maps where typing in a location and hitting enter either does nothing or just says “Loading…” forever, you’ve very likely hit upon a bug in the Skype click-to-call extension.

Skype automatically installs this “add-on” when you install Skype and it adds links to phone numbers so that you can click them to call on Skype. Unfortunately, Google Maps seems to be triggering a bug in the extension and so it is either corrupting the data returned to the Google Maps javascript or just preventing the request from ever returning.

To solve the issue, just uninstall the Skype click to call add-on. I did that by right clicking the Skype icon in the toolbar and selecting “Uninstall” but it should be possible to remove it from the about:extensions page.

I also found a Skype click-to-call plugin in about:plugins that I disabled as well – I’m not sure if it’s removed along with the extension.

More Career Advice

2011-12-11T17:44:41Z

Following on from my last post, Jason Adam Young has some excellent advice to help you continually get better and better and building software and perhaps more importantly, be more and more valuable to whoever you happen to be working for.

Career Progression in Technology

2011-12-06T18:45:15Z

This afternoon during an interview, a potential new hire at LMAX asked what LMAX could offer in the way of career progression. Since I’ve been thinking a fair bit about what comes next in my career now that I’m moving back to Australia I thought I had a pretty good answer and that it might be worth sharing here2.

In most industries career progression is signified by ever fancier sounding job titles, but for a software developer, the fancier the title sounds the less likely you are to actually do any development. Essentially, career progression often amounts to career migration – into “architecture”1 or managements. If you really love developing software though you need to look at it a different way. If I spend the rest of my career as a “Senior Software Engineer” I’ll be pretty happy, so long as I can keep honing my skills and becoming more valuable to the companies I work for. Ideally I’d like the pay I take home to grow in line with that value.

Given that world view, the best way I’ve found to progress my career as a software engineer is to work in a place where I’m surrounded by people smarter than I and where I regularly feel stupid. Feeling stupid is a pre-requisite to really learning – it means you’ve just found an area where you have a large gap in your knowledge. If you also work with people smarter than yourself that becomes an opportunity to learn from them.

What’s particularly amazing about this process – and this is something that working at LMAX has really taught me – is that if you spend enough time being excited by the learning opportunities that feeling stupid highlights, you find yourself suddenly providing the knowledge to teach other people who are feeling stupid. So now you get the satisfaction of feeling stupid and learning something new, plus the opportunity to demonstrate how much you’ve learnt by sharing that knowledge with others.

If that’s not an awesome way to spend your career, I don’t know what is3.

1 – read, powerpoint production↩

2 – I think I was much more succinct when speaking off the cuff ↩

3 – and if you happen to have that kind of environment and are looking for new employees, my CV is online ↩

Looking for Work In Australia

2011-11-27T20:54:49Z

After four years living in the UK, my wife and I have decided to move back to Australia mid-February 2012 to be closer to family. As such, I’m now starting to look at job opportunities back in Oz. If you're hiring, I'd love to hear from you and discuss how we might work together. My CV is available online to give you an idea of my experience and of course the backlog of this blog shows some of my thinking and learning throughout the years1.

My one key criteria is that I'm looking for a role that provides lots of challenges and opportunities to learn. I'm not particularly concerned with what language or the particular technologies used, but I do believe strongly that Agile methods, automated testing, continuous delivery and a focus on quality are essential to success.

If you think you could use someone like me or have any questions feel free to drop me an email.

1 – as well as plenty of my mistakes but that’s just part of learning↩

Simple Long Poll Patterns in JavaScript

2011-11-13T20:10:11Z

A couple of little patterns for writing long-poll based JavaScript applications have evolved in some internal apps I’ve been working on lately so I thought I’d document them for posterity.

Simple Long Poll Loop

(function($) {
    var longPollUrl = '/longpoll';

    var lastReceivedSequence;

    function poll() {
        $.ajax(longPollUrl, {
            data: { 'lastSequence': lastReceivedSequence },
            dataType: 'json',
            timeout: 15000,
            success: function(data) {
                if (!data) return;
                lastReceivedSequence = data.sequenceNumber;
                $(window).trigger('onLongPollEvent', data);
            },
            complete: poll
        });
    }
})(jQuery);
 view raw basic.js This Gist brought to you by GitHub.

There’s very little special about this – it basically just uses jQuery to do a standard long poll. The pattern that I like is that instead of calling a particular function directly to handle events it just uses the jQuery trigger function to fire an event that can be handled by any interested code, without introducing tight coupling between the two bits of code. Listeners would receive events with:

$(window).bind('onLongPollEvent', function(eve, longPollData) { ... });
 view raw listener.js This Gist brought to you by GitHub.

Automatically Reload JavaScript Changes

One of the internal tools I’ve been working on is our “big feedback” page for reporting build results. This page is displayed on a large monitor the whole team can see, so refreshing the page to load JavaScript changes requires actually getting up and walking over to it. Fortunately, there’s a simple pattern to get the JavaScript to reload itself:

var serverVersion;
...
if (data.serverVersion != serverVersion) {
    window.location.reload();
    return;
}
...
 view raw reload.js This Gist brought to you by GitHub.

Each long poll event sent down to the browser includes the server version – when it changes, the JavaScript reloads the page. There are two nice little details here:

Deploying a new version actually requires some downtime while the server restarts. Naive implementations will reload the page when the long poll request returns an error but that usually results in the page attempting to reload and also getting an error or no response from the server and now there’s no JavaScript running to recover. Instead, this will wait for the server to come back up and be answering requests again before reloading.
So I don’t have to remember to update the server version, the server actually sends the UNIX time stamp from when it started up. Anytime the server restarts, that value changes and the JavaScript will reload. This can lead to some false positives but it’s extremely simple and effective. Another option would be for the build scripts to insert the SVN revision but that doesn’t work as well for development.

Usually I insert this reload code directly into the long poll loop so that it is guaranteed to run first and avoid any potential problems with data formats changing. Even the serverVersion data format can change since it doesn’t try to parse it in any way – any change, including data.serverVersion no longer being present, will cause the page to reload.

Put together the code comes out something like:

(function($) {
    var longPollUrl = '/longpoll';

    var lastReceivedSequence;
    var serverVersion;

    function poll() {
        $.ajax(longPollUrl, {
            data: { 'lastSequence': lastReceivedSequence },
            dataType: 'json',
            timeout: 15000,
            success: function(data) {
                if (!data) return;
                lastReceivedSequence = data.sequenceNumber;
                if (data.serverVersion != serverVersion) {
                    window.location.reload();
                    return;
                }
                $(window).trigger('onLongPollEvent', data);
            },
            complete: poll
        });
    }
})(jQuery);
 view raw full.js This Gist brought to you by GitHub.

Nothing earth-shatteringly new, but handy little tools to have up your sleeve.

Background Logging with the Disruptor

2011-09-26T20:40:15Z

Peter Lawrey posted an example of using the Exchanger class from core Java to implement a background logging implementation. He briefly compared it to the LMAX disruptor and since someone requested it, I thought it might be interesting to show a similar implementation using the disruptor.

Firstly, let’s revisit the very high level differences between the exchanger and the disruptor. Peter notes:

This approach has similar principles to the Disruptor. No GC using recycled, pre-allocated buffers and lock free operations (The Exchanger not completely lock free and doesn't busy wait, but it could)
Two keys difference are:
there is only one producer/consumer in this case, the disruptor supports multiple consumers.
this approach re-uses a much smaller buffer efficiently. If you are using ByteBuffer (as I have in the past) an optimal size might be 32 KB. The disruptor library was designed to exploit large amounts of memory on the assumption it is relative cheap and can use medium sized (MBs) to very large buffers (GBs). e.g. it was design for servers with 144 GB. I am sure it works well on much smaller servers. ;)

Actually, there’s nothing about the Disruptor that requires large amounts of memory. If you know that your producers and consumers are going to keep pace with each other well and you don’t have a requirement to replay old events, you can use quite a small ring buffer with the Disruptor. There are a lot of advantages to having a large ring buffer, but it’s by no means a requirement.

It’s also worth noting that the Disruptor does not require consumers to busy-spin, you can choose to use a blocking wait strategy, or strategies that combine busy-spin and blocking to handle both spikes and lulls in event rates efficiently.

There is also an important advantage to the Disruptor that wasn’t mentioned: it will process events immediately if the consumer is keeping up. If the consumer falls behind however, it can process events in a batch to catch up. This significantly reduces latency while still handling spikes in load efficiently.

The Code

First let’s start with the LogEntry class. This is a simple value object that is used as our entries on the ring buffer and passed from the producer thread over to the consumer thread.

Peter’s Exchanger based implementation – the use of StringBuilder in the LogEntry class is actually a race condition and not thread safe. Both the publishing side and the consumer side are attempting to modify it and depending on how long it takes the publishing side to write the log message to the StringBuilder, it will potentially be processed and then reset by the consumer side before the publisher is complete. In this implementation I’m instead using a simple String to avoid that problem.

import com.lmax.disruptor.EventFactory;

class LogEntry
{
    public static final EventFactory<LogEntry> FACTORY = new EventFactory<LogEntry>()
    {
        public LogEntry newInstance()
        {
            return new LogEntry();
        }
    };

    long time;
    int level;
    String text;
}

 view raw LogEntry.java This Gist brought to you by GitHub.

The one Disruptor-specific addition is that we create an EventFactory instance which the Disruptor uses to pre-populate the ring buffer entries.

Next, let’s look at the BackgroundLogger class that sets up the process and acts as the producer.

import com.lmax.disruptor.RingBuffer;
import com.lmax.disruptor.dsl.Disruptor;

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class BackgroundLogger
{
    private static final int ENTRIES = 64;

    private final ExecutorService executorService;
    private final Disruptor<LogEntry> disruptor;
    private final RingBuffer<LogEntry> ringBuffer;

    BackgroundLogger()
    {
        executorService = Executors.newCachedThreadPool();
        disruptor = new Disruptor<LogEntry>(LogEntry.FACTORY, ENTRIES, executorService);
        disruptor.handleEventsWith(new LogEntryHandler());
        disruptor.start();
        ringBuffer = disruptor.getRingBuffer();
    }

    public void log(String text)
    {
        final long sequence = ringBuffer.next();
        final LogEntry logEntry = ringBuffer.get(sequence);

        logEntry.time = System.currentTimeMillis();
        logEntry.level = level;
        logEntry.text = text;

        ringBuffer.publish(sequence);
    }

    public void stop()
    {
        disruptor.shutdown();
        executorService.shutdownNow();
    }
}



 view raw BackgroundLogger.java This Gist brought to you by GitHub.

In the constructor we create an ExecutorService which the Disruptor will use to execute the consumer threads (a single thread in this case), then the disruptor itself. We pass in the LogEntry.FACTORY instance for it to use to create the entries and a size for the ring buffer.

The log method is our producer method. Note the use of two-phase commit. First claim a slot with the ringBuffer.next() method, then copy our values into that slot’s entry and finally publish the slot, ready for the consumer to process. We could have also used the Disruptor.publish method which can make this simpler for many use cases by rolling the two phase commit into call.

The producer doesn’t need to do any batching as the Disruptor will do that automatically if the consumer is falling behind, though there are also APIs that allow batching the producer which can improve the performance if it fits into your design (here it’s probably better to publish each log entry as it comes in).

The stop method uses the new shutdown method on the Disruptor which takes care of waiting until all consumers have processed all available entries for you, though the code for doing it yourself is quite straight-forward. Finally we shut down the executor.

Note that we don’t need a flush method since the Disruptor is always consuming log events as quickly as the consumer can.

Last of all, the consumer which is almost entirely implementation logic:

import com.lmax.disruptor.EventHandler;

public class LogEntryHandler implements EventHandler<LogEntry>
{

    public LogEntryHandler()
    {
    }

    public void onEvent(final LogEntry logEntry, final long sequence, final boolean endOfBatch) throws Exception
    {
        // Write
    }

}

 view raw LogEntryHandler.java This Gist brought to you by GitHub.

The consumer’s onEvent method is called for each LogEntry put into the Disruptor. The endOfBatch flag can be used as a signal to flush written content to disk, allowing very large buffer sizes to be used causing writes to disk to be batched when the consumer is running behind, yet also ensure that our valuable log messages get to disk as quickly as possible.

The full code is available as a Gist.