David Winterbottom (@codeinthehole)

Octopus Energy

Last Wednesday was my last day at Yoyo Wallet. Thursday marked my first day at Octopus Energy.

I'm deeply excited about Octopus. Through innovative use of technology, especially automation and cloud computing, Octopus Energy is going to radically change how people consume and pay for energy in the UK. We're going to be both cheaper and massively better at customer service than the much-maligned incumbents.

Right now, we're looking to build a superb tech team. Get in touch if you're looking for a challenge and would like to play a formative part in a company that's genuinely making the world a better place, in helping transition the UK to a lower carbon future.

A useful template for commit messages

Here's a useful heuristic for writing better commit messages. Set your commit message template to:

# If applied, this commit will...

# Explain why this change is being made

# Provide links to any relevant tickets, articles or other resources

and you'll be guided into writing concise commit subjects in the imperative mood - a good practice. See rule 5 of Chris Beam's "How to write a commit message" for the inspiration of this tip and more reasoning on the use of the imperative mood.

To do this in Git, save the above content in a file (eg ~/.git_commit_msg.txt) and run:

$ git config --global commit.template ~/.git_commit_msg.txt

Here's what this looks like in practice:

Try it - it's ...

Copying Postgres output into a spreadsheet

I often need to grab information from a Postgres database and paste it into a spreadsheet for sharing with others. Google Sheets needs the pasted data to be tab-separated in order to be correctly split into columns. This isn't the default behaviour for psql but here's how to configure psql's output to get it.

At a psql prompt, switch to unaligned output

=> \a

and set the field separator to a tab character:

=> \f '\t'

then the output from subsequent SELECT ... statements can be cleanly pasted into your Google Doc.

An SSH tip for modern AWS patrons

Cloud computing and immutable infrastructure deployments have changed the way I use SSH. I miss the days when I could run:

$ ssh app1-prod

to jump onto a machine and investigate an issue. This would work as, back in the days of yore, your web servers didn't change IP address several times a week so I could create a helpful alias in ~/.ssh/config:

Host app1-prod
    User example_user
    HostName 74.207.251.29

This circumvented the labour-intensive act of typing in the remote username and IP address when SSHing around town.

I can no longer do this as:

Immutable infrastructure deployments mean EC2 instances are replaced for every update so the IP addresses keep changing. Life is too short to ...

commandlinefu.com is in new hands

For the record, I no longer maintain commandlinefu.com. I've handed over the baton to notable Bay Area celebrity, Jon Hendren (@fart). This is a good thing for the site and its users as I have been unable to do much maintenance in recent times. I look forward to seeing how the site evolves.

Backing up Postgres database rows before deleting them

Hmmm, this delete statement is taking longer than I thought it would...

If you ever have to manually run a SQL delete statement in psql, you can back-up the rows you're about to delete:

\copy ( select * from $table where id in ( ... ) ) to '/tmp/backup.csv'

and, if you've got your filter clause wrong (we've all done it), restore them with:

\copy $table from '/tmp/backup.csv'

Moderately useful.

Avoiding clashing Django migrations

Managing South migrations on a multi-developer Django project can be painful. Developers working on separate branches will often create migrations for the same app with the same migration number [1]. When merged into master, these clashing migrations can cause deployment hiccups as South will complain if migrations are applied out of order.

There are various techniques available for dealing with this [2], but what we do at Yoyo is test for such clashes as part of our Travis continuous integration.

This is done by calling a makefile target from .travis.yml:

# .travis.yml

language: python

python:
    - 2.7

install:
    - make virtualenv

script
    - make test
    - make migration_test

where the migration_test target is:

# makefile

MIGRATION_CLASHES=$(shell find . -type f -name "*.py" | grep ...

Bootstrapped virtualenvs

The excellent virtualenvwrapper supports a postmkvirtualenv script to bootstrap your virtual environments. Here's a useful implementation:

#!/usr/bin/env bash

# Grab project name from virtualenv name
NAME=$(basename $VIRTUAL_ENV)

# Set terminal title on postactivate
echo "title $NAME" > $VIRTUAL_ENV/bin/postactivate

# Change directory to root of project on postactivate. We assume the
# mkvirtualenv is being run from the root of the project. This line
# will need to edited later if not.
echo "cd $PWD" >> $VIRTUAL_ENV/bin/postactivate

# Run postactivate now to get the title set
source $VIRTUAL_ENV/bin/postactivate

This ensures that a new virtualenv has a postactivate script which:

Sets the terminal title to that of the virtualenv
Changes directory to the root of the project

By convention, such ...

Integrating Django application metrics into Zabbix

At Tangent, we use Zabbix for monitoring and alerting. This is a note-to-self on how to configure application monitoring.

Management command

You need a script that prints out a value to STDOUT. A simple management command suffices:

from django.core.management.base import BaseCommand, CommandError

class Command(BaseCommand):

    def handle(self, *args, **options):
        if not args:
            print self.usage()
            return
        method_name = 'fetch_%s' % args[0]
        if not hasattr(self, method_name):
            raise CommandError("No method found with name '%s'" % method_name)
        print getattr(self, method_name)(*args[1:])

    def usage(self):
        fetchers = [m for m in dir(self) if m.startswith('fetch')]
        descriptions = []
        for fetcher in fetchers:
            method = getattr(self, fetcher)
            docstring = method.__doc__.strip() if method.__doc__ else "<no description>"
            descriptions.append ...

Linking to Github

It was rightly pointed out yesterday that it's dangerous to link to lines or blocks of code on Github without explicitly specifying the commit hash in the URL. On this theme, consider this git command:

$ git browse -u -- commit/$(git rev-parse HEAD)
https://github.com/tangentlabs/django-oscar/commit/17851d4b66922f8d7e203e2b469040690c84a0db

This emits the Github URL to the HEAD commit on the current branch, specifying the commit hash in the URL. Note that the browse subcommand is provided by the excellent hub library.

Pasting links to commits is common, both for mailing list posts and within discussion on Github itself. Getting the correct URL quickly is useful.

We can streamline further using an alias:

# ~/.gitconfig

[alias]
url = !hub browse -u -- commit ...

Continuously rebuild your project

New developers joining a project will often find that the project won't build cleanly on their machine, and hours of time will be sunk into setting up the project so work can start. This is sad and expensive for all concerned.

This is a particular menace in agencies (or anywhere with lots of small projects) where a large team of developers need to jump between projects. Tools like Vagrant and Docker can help but aren't the panacea they first seem to be [*].

Counter this by using continuous integration to build your project from scratch. Then any changes that break the build process (such as database schema changes not applying correctly) will be spotted early. New team members will ...

Using the silver searcher with Vim

If you're not doing this already, then you should use the Silver Searcher within Vim for rapid, convenient file searching. In a nutshell, ag offers similar functionality to ack but with much better performance.

It's easily installed - on OSX, run:

$ brew install the_silver_searcher

Urge Vim to use it for :grep commands by adding the following to ~/.vimrc:

if executable('ag')
    " Note we extract the column as well as the file and line number
    set grepprg=ag\ --nogroup\ --nocolor\ --column
    set grepformat=%f:%l:%c%m
endif

:grep searches are now lightning-fast and respectful of your ~/.gitignore patterns. It's genuinely impressive.

Improve your efficiency further by remapping the keys for jumping through search matches (stored in the "quickfix ...

The road to Oscar 1.0

Oscar 0.7 was released this week, a comparatively minor house-keeping release that allowed us to reduce our pull-request and issue backlogs. We have a plan for Oscar through 2014 and this article outlines the roadmap.

v1.0

We're aiming to be v1.0 before the end of the year [*]. Oscar has been beta for nearly two years and its APIs are stabilising. We're ready to commit to the backward compatibility responsibilities associated with coming out of beta.

Plus, this will help Tangent Snowball (who sponsor Oscar's development) promote Oscar to blue-chip or "enterprise" clients, who feel nervous enough using open-source e-commerce software, let alone beta open-source software.

There isn't a lot to do before this ...

Command-line tips for effective release announcements

We finally released Oscar 0.6 last week. The process brought home the importance of writing your release notes as you go rather than at the end. It's a real pain to extract the key changes from 1200 commits spread over the last 8 months. Lesson learnt.

This article is largely a note-to-self in case I have to repeat the process. However, if you do find yourself in a similar position, here are a few command-line tricks for analysing your git history.

Analysing codebase changes since a tag

Basics: browse commits since the last tagged release

$ git log 0.5..0.6

The --name-status option for git diff is useful for analysing codebase changes between two commits. For instance ...

How to install PostGIS and GeoDjango on Ubuntu

Despite its extensive documentation, getting GeoDjango installed and configured can be a pain. Here are my notes for future reference:

Installation on Ubuntu 12.04

First, ensure your system locale is UTF8 as PostgreSQL uses it to determine its default encoding during installation:

$ export LANGUAGE="en_US.UTF-8"
$ export LANG="en_US.UTF-8"
$ export LC_ALL="en_US.UTF-8"

Now install dependencies:

$ sudo apt-get update
$ sudo apt-get install postgresql-server-dev-9.1 postgresql-9.1-postgis

PostgreSQL should now be installed and running with UTF8 encodings. Verify this with:

$ sudo -u postgres psql -l
                                     List of databases
       Name       |  Owner   | Encoding |   Collate   |    Ctype    |
------------------+----------+----------+-------------+-------------+-...
 postgres         | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 template0        | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
                  |          |          |             |             |
 template1        | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
                  |          |          |             |             |

Now create a spatial database ...