The Daily Build

3 Easy Ways to Stick to a Coding Standard

2009-08-25T19:26:27Z

When you’re writing python, you don’t need a lot of debate over the minutiae of most coding standards. PEP 8 does that for you. Even better, there are some tools that make it really easy to stick to the standard.

Why do this? Well, for one thing it makes code reviews easier when everyone follows the same conventions. It also makes maintenance easier.

pep8.py is a style checker that enforces the rules of PEP 8. The “official home” (?) at browsershots.org was dead as I was writing this. (Thanks GitHub!) Run pep8.py on your code and it will tell you where you’ve drifted from the standard.
pylint is lint for python. It provides more functionality than pep8.py, but is not a strict superset. (pep8.py is pickier about whitespace issues.) Pylint is also configurable to enforce various naming rules. Like most lints, the SNR is pretty low, but you can turn off most of the noise and get a reasonable signal for the things you want to check.
Subversion (as well as most other tools) can be configured to run a script every time you check in code. Run one or both of the above tools in the pre-commit hook and bad code will be rejected. (I’d be wary of doing this with pylint unless you’ve got the categories of “noisy” warnings turned off.) I don’t have anything cookbook for this yet, but I’m working on it…

Five Days to a Django Web App: Day Three, Coding Thanks for coming back for...
Python Exception Handling: Cleanup and Reraise I’ve had this code around...
Makefiles are Software Too This post was inspired by...

Hassle Free Way to Kill Sudo’d Jobs

2009-08-12T15:53:10Z

Every now and then I have to run a foreground job under sudo that doesn’t want to die when I hit ^C. Then it’s a hassle to ^Z, get the pid of the sudo job, and sudo kill that pid.

So I wrote a little script (or a template for scripts) that runs the sudo job in the background (but preserves stdout/stderr) and relies on bash to clean up the job when you ^C the script. Only gotcha with this is that you may have to retype your sudo password when you ^C if your authentication has timed out by the time you get around to killing it.

#!/bin/bash

function cleanup()
{
    sudo kill $job_pid
    wait $job_pid
    exit 0
}

trap cleanup SIGTERM
trap cleanup SIGINT

sudo long_running_foreground_process &
job_pid=$!
wait

Python Exception Handling: Cleanup and Reraise I’ve had this code around...
Using Python’s ctypes to Call Into C Libraries The ctypes module makes loading and...
Makefiles are Software Too This post was inspired by...

Using Python’s ctypes to Call Into C Libraries

2009-08-05T16:23:24Z

The ctypes module makes loading and calling into a dynamic library incredibly easy:

>>> from ctypes import CDLL
>>> libc = CDLL('libc.so.6')
>>> print libc.strlen('abcde')
5

As with everything else in python, it gets even better when you scratch the surface. In the example above, CDLL returns an object that represents the dynamic library. You can access the functions in that library by attribute access (”libc.strlen”) or item access (”libc['strlen']“). Both access mechanisms return a callable object.

This callable object has an “errcheck” attribute that can be assigned a callable. We can use this for error-checking our calls into the library. Let’s write a simple version of the “kill” command that uses the kill(2) system call.

import sys
from ctypes import *

# Load the library.
libc = CDLL('libc.so.6')

# Our error checking function. This will receive the
# return value of the library function, the function that
# was called, and the arguments passed to the function as a
# tuple.
def kill_errcheck(retval, func, funcargs):
    '''Check for error -- retval == -1.'''
    if retval < 0:
        raise Exception('kill%s failed' % (funcargs, ))
    return True

# Get the kill function from the standard library.
kill = libc.kill

# Set the error checker for kill().
kill.errcheck = kill_errcheck

# Pass the command line argument as a pid to kill, with
# SIGSEGV (11).
pid = int(sys.argv[1])
kill(pid, 11)

Save this as kill.py. Then, in your shell, try something like this:

# Notice that the 3401 is the pid of the process
# we're putting into the background. Yours will
# be different.
bash$ sleep 120&
[1] 3401
bash$ python kill.py 3401
[1]+ Segmentation Fault         sleep 120
bash$ python kill.py 3401
Traceback (most recent call last):
  File "kill.py", line 17, in
    kill(pid, 11)
  File "kill.py", line 10, in kill_errcheck
    raise Exception('kill%s failed' % (funcargs, ))
Exception: kill(3401, 11) failed

At line 4 of the output we run sleep in the background. At line 5 we learn the pid of this process. At line 6 we run our kill program, giving it the pid we just spawned, and we see the notification from bash that the process was killed (with signal 11, segmentation fault). At line 8 we run our kill program again on pid 3401, but it doesn’t exist, the kill system call returns -1, and our error checker raises an exception when it detects the system call failure.

But wait, there’s more… I’m working on a follow up post that combines ctypes.Structure with calls into a linux system call.

Python Exception Handling: Cleanup and Reraise I’ve had this code around...
Five Days to a Django Web App: Day Three, Coding Thanks for coming back for...
Jesse Noller on Python Jesse Noller has been republishing...

Yet Another Python Enum Module

2009-08-05T03:25:13Z

I didn’t like the existing enum recipes, so I cooked up what I feel is a better way of working with enumerations in python. The result is yapyenum, hosted on github. Rather than come up with something new to say about it, I’ll just repost the README here:

This module provides named enumerations for python.

Unlike other implementations that I have seen, this is *not* an anonymous enumeration. To use it derive from the Enumeration class provided in enum.py and list the names of the enum members in the class member _enum_.

The enum is itself a singleton class. It tries to be immutable via __slots__ and by refusing to allow its members to be changed.

Enum members are an integer subclass with a “name” property and that knows how to pretty print itself. The enum values are interchangeable with integers, which may or may not be what you want.

The class supports membership (”FOO in MyEnum”) and mapping a non-instance-member value to a name. (I.e. if FOO uses the integer value 1, then MyEnum.name(1) returns “FOO”.)

Tested on 2.4.6, 2.5.2, 2.6.2, on a combination of debian sarge, etch, lenny, and ubuntu 9.04.

There are obvious capabilities that could be added but this does all that I need so far. Patches will be happily accepted.

Permissive license (MIT). Have fun.

For other enum recipes, see:

PEP 354 (rejected enum proposal for python)
The enum module in pypi
Recipe 413486: First Class Enums in Python
Recipe 305271: Enumerated values by name or number
Recipe 67107: Enums for Python
Recipe 81098: python enum with strings
At Stack Overflow: What’s the best way to implement an ‘enum’ in Python?
The Python Cookbook (at page 607 on Google Books)

Several of these recipes have comments along the lines of “if you’re using one of these too-fancy enum recipes in python, you’re doing something unpythonic”. This is likely true, but sometimes unavoidable for social, political, and/or historical reasons.

Jesse Noller on Python Jesse Noller has been republishing...
Python Exception Handling: Cleanup and Reraise I’ve had this code around...
Five Days to a Django Web App: Day Three, Coding Thanks for coming back for...

Python Exception Handling: Cleanup and Reraise

2009-08-05T03:27:15Z

I’ve had this code around for a while and had an opportunity to drag it out the other day and dust it off. The problem: Every now and again there’s a situation where you don’t really want to catch an exception, but you do want to perform some cleanup and let the exception propagate up the stack. Sometimes there’s an extra wrinkle in that the cleanup code may itself throw an exception (that I’m simply going to assume we can ignore).

You can run the file to see the behavior. Simply provide an integer 1-5 as a command line argument and you’ll run the selected scenario and see the output. The goal in this case is for cleanup to occur and an exception to be reported as having occurred at line 10.

The code in reraiser1 is wrong because this behaves as if a brand new exception were thrown at line 27. That may not seem so bad, but this code is pretty simple. If this happens and the stack trace is deep, it will be almost impossible to diagnose what went wrong.

The code in reraiser2 shows what happens when a second exception occurs in the except block. A bare raise statement here might be an attempt to re-raise the original exception, but python’s rules about re-raising specify that the most recent exception in the scope is what is reraised. In this case, that’s the exception thrown from the cleanup function. Again, this makes troubleshooting difficult.

In reraiser3 I worked around the problem in reraiser2 by moving the cleanup function’s exception into a separate scope by defining a local cleanup function and calling it from within the except block. This prevents the cleanup function’s exception from polluting the scope with an irrelevant exception and we can re-raise the original exception. This results in a stack trace rooted at line 10.

Reraiser4 takes a different approach. Instead of moving the cleanup function’s exception into a separate scope, it captures the traceback information from the original exception and then passes it back to the raise statement so that the reported traceback is accurate.

The cleanest way to handle this is to use a finally block as shown in reraiser5. This situation is what “finally” is meant for: it does not trap the exception, it just gives you a chance to clean up before control moves back up the stack to the caller. The presence of finally clues readers in to the fact that you aren’t messing with the exception, and that the point of the block is to perform cleanup.

Kindly drop me a note if I’ve got something wrong above, or if I’m missing a technique (or a common anti-pattern!). Thanks.

Using Python’s ctypes to Call Into C Libraries The ctypes module makes loading and...
Jesse Noller on Python Jesse Noller has been republishing...
Five Days to a Django Web App: Day Three, Coding Thanks for coming back for...

Makefiles are Software Too

2009-04-14T03:17:48Z

This post was inspired by recent experience with some horrible build scripts from the open source world — but I’ve seen enough in-house badness over the years that I wanted to establish some basic parameters for build scripts.

This is a really broad way to divide the world, but I see two important categories of build scripts. (By “build script” I mean Makefiles, SConsfiles, Ant files, and their ilk).

Build scripts intended for in-house use, never seen by outsiders.
Build scripts that you distribute to external users.

Both sets of build scripts are important. Let’s consider the build script (or Makefile system, or whatever you’re using) as a software system independent of the software that is being built. Its primary requirement is to reliably convert your source code into an executable. Any decent build script has numerous other requirements, most of which are probably implied. I have never seen a written requirement for the numerous build scripts I’ve written. I have only rarely heard them discussed — and never actually posed as “requirements”.

I have, always taken the following meta-requirements for granted. Based on my experience with some fairly awful build scripts, I guess these aren’t universally acknowledged.

The build script should build all the “normal” executables with a single command.
The build script must encapsulate all environment variables within the script. (This is a corollary of the above.)
The build script should not unnecessarily rebuild source code in a directory tree that has previously been built. (This is make’s raison d’etre.)
The build script should be documented. It doesn’t have to be elaborate, but a five-line comment at the top of the script describing the available command-line variables would be nice.
make clean or its equivalent must work reliably.
It should rarely be necessary to run make clean.
Bonus: the build should be parallelizable, to be able to take advantage of multicore machines and/or distributed builds.

Back to those two categories. If you screw up an in-house build script, it’s primarily your team that’s going to suffer. I’m not aware of any bugs from SQA or Customers on projects I’ve been on that were traced back to a build script, but I can see where it could happen. Mainly what I’ve seen is developers habitually wasting time rebuilding source that should not need to be rebuilt. The worst offender in this respect was a build script that forced a “make clean” at the beginning of every build. In the immortal words of Dave Barry, I’m not making this up. Really.

(Actually, I think it was even worse when I was dealing with a build script that required a “make clean” every time around but didn’t have it coded into the script!)

I can’t say I have a completely clean conscience in this area: I wrote a big pile of Makefiles for a past employer in which I unknowingly used features (bugs?) of a specific version of GNU Make that were “fixed” in a subsequent version. As time went on it became (a) a much bigger task to fix the Makefile code that used the “feature” and (b) harder to keep the specific version of GNU Make that worked with the Makefiles! I imagine they’re still stuck with this situation… sorry guys.

In the second category, when you toss a slapped-together build script to your unsuspecting users you do all of human society a disservice. Now they are not only wasting their time fighting with a bad build script, but they often have the disadvantage of being unable to contact the author directly for support or commiserate with other experienced users of the system.

So do us all a favor: if you don’t know what you’re doing when you start to write your next build script, please ask for help. Otherwise we’ll have to resort to some sort of professional licensing scheme for build script authors.

The Toolkit of a Software Engineer This is a rundown of...
The cost of (not) testing software Great post on the cost...
3 Easy Ways to Stick to a Coding Standard When you’re writing python, you...

Five Days to a Django Web App: Day Four, Deployment

2009-08-25T22:46:16Z

Thanks for your patience, and for coming back for a discussion of deploying our Django web app.

In case you missed any of the previous posts in this series, here they are:

Day One, Get Ready (Concept and prep)
Day Two, Mockups (Creating a design)
Day Three, Coding (Coding tests, views, templates, and models)

Pre-Deployment

First, we need to make a couple of decisions:

How are we going to push updates to the live site: FTP, git, svn?
How are we going to handle backups?

Version Control as Distribution System

In my case, I’m using svn+ssh to push updates. Notice that this does not require special setup on your server — you do not need to install the svn stuff that DreamHost or your host may provide. Just do svn init to create a repository on the server (outside the DocumentRoot!). Then point your development pc to svn+ssh://USERNAME@host.example.com/home/USERNAME/svn/PROJECT. (Git works similarly, no support from your host required except the binaries.)

In the directory on the host where you’re going to store your project files, point to the same URL. (You could use the file:///…/ url, but I prefer to avoid accessing the repo directly. Superstition?)

Now, whenever you make a change on your development system, just "svn ci" and then on the host “svn up” and restart your fcgi to pick up the new code. Presto! The live site is updated with your change.

Backups

You must have a backup strategy: Your app will have users. Your host’s disk will burp. Your users will hate you when the disk burps and you don’t have a good backup.

There are probably 374 different ways of backing up your Django app. The two major things you need to capture are the database and your code. If you are storing objects (e.g. uploaded files) outside the database, you’ll need to back these up too. The option I’m using is django-backup.

Pull the code from subversion into your project. Rename the directory to "django_backup". Add django_backup to your INSTALLED_APPS. Verify it works by running ./manage.py backup -c. Sanity check the backup by doing zless backups/*.gz. We’ll set up a cron job on the host to run this regularly when we deploy.

Deploy

I’ve previously written about deploying Django apps on DreamHost, so I’m not going to duplicate that here. Keep in mind that you want to use the version control strategy outlined above. Read that article, deploy your app and come back here when you’re done.

Post-Deployment

Backup

Let’s add that cron job we previously mentioned. On the host, run crontab -e. If you’re on DreamHost and this if the first time you’ve used cron, it will prompt you for an email address to send output to. Then it will dump you into an editor. (Side note: "joe" is the default. If you want something different, like vim, be sure that EDITOR=/usr/bin/vim is set in your environment.)

Set up a job something similar to the following:


MAILTO="YOU@EXAMPLE.com"

# m h  dom mon dow   command
4 2 * * * (cd /home/PATH/TO/PROJECT; ./manage.py backup -c --email=YOU@EXAMPLE.com)

This will run a backup every day at 02:04 (AM). You will get two emails: one with the output from the job, and one with the compressed backup file. (When you get to the point where your database backups are too big for email, you’ll need to find another strategy.)

Now we’ve got another problem to solve: we’re going to accumulate a bunch of backups on the disk. Let’s get rid of the old backups. This is pretty safe, since we’re receiving backup files via email. Add a cron job like this:


14 2 * * * (cd /home/PATH/TO/PROJECT;
    touch --date=`date --iso --date='10 days ago'` .backup.oldest;
    find ./backups/ -mindepth 1 \! -newer .backup.oldest -execdir rm '{}' +)

(Formatted here for readability — you need to put that all on one line.)

This will remove backup files older than 10 days. You could do this more concisely with a tool like tmpwatch or tmpreaper, but neither is installed on my host and this incantation should work on pretty much any flavor and installation of linux.

At this point we’re deployed and the majority of the work is done. Tomorrow we’ll take a look at some maintenance issues.

Five Days to a Django Web App: Day One, Get Ready This is the first in...
Five Days to a Django Web App: Day Three, Coding Thanks for coming back for...
Five Days to a Django Web App: Day Two, Mockups Welcome back. Ready for Day...

Firefox Quick Search for Google Maps

2009-02-18T13:46:14Z

I recently found myself wanting to quickly figure out how far (both miles and driving time) it is to various destinations. At first I was keeping a Firefox tab open to Google Maps and kept keying in the address for the destination. Then I realized that I could make a “Quick Search” and get the info faster. Add this link to your Quick Searches folder:

http://maps.google.com/maps?f=d&saddr=Elm+St,+ Manchester,+NH+03101&daddr=%s&output=html

[Formatted here for readability. Put it all on one line with no spaces.]

Edit the properties so that the “saddr=” part is your start address. Change the keyword to “gm”. This is set to only output simple HTML (no map). If you want the full map, remove the &output=html on the end.

Now, when you need to get a quick idea of how long it is going to take you to get somewhere, open a new tab and type “gm city, state”.

Five Days to a Django Web App: Day Four, Deployment Thanks for your patience, and...
Code Review Tools Yesterday I posted twenty reasons...
Getting Started in C Some may argue that C...

Five Days to a Django Web App: Day Three, Coding

2009-02-27T17:55:59Z

Thanks for coming back for Day Three!

[Note: Sorry this post is a day late. It was all ready to go late yesterday, but some of the code included below triggered a bug either in WordPress or ScribeFire and the whole post got mangled. I managed to resurrect it today from drafts, and I think it's coherent, but if you find some problem with it please drop me a note.]

Progress So Far

Yesterday we built some mockups, just HTML and CSS — nothing active. Hopefully you’ve had a chance to run your mockups past a couple of people for feedback and ideas.

Armed with these mockups, we’re ready to get started coding. Today’s post is the longest in the series. Take it in chunks if you need to. (I didn’t write it all at once either.)

Foundation

MySQL

First, go to your web host’s control panel and create two MySQL databases for this project. I created “resumandb” and “test_resumandb”. This latter database is needed for running the built-in test system.

You’ll get an error/warning if the test database exists when you first run tests. However, the control panel needs to set user privileges and it seems like the only way it knows how to do this is by creating the database.

Set up the same databases on your local system:


bash$ mysql -u root
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1969
Server version: 5.0.32-Debian_7etch8-log Debian etch distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> CREATE DATABASE resumandb;
Query OK, 1 row affected (0.00 sec)

mysql> GRANT ALL ON resumandb.* TO 'resuman'@'localhost' identified
    by 'reallY_baD_9passworD!';
Query OK, 0 rows affected (0.11 sec)

mysql> GRANT ALL ON test_resumandb.* TO 'resuman'@'localhost' identified
    by 'reallY_baD_9passworD!';
Query OK, 0 rows affected (0.0 sec)

Notice that you don’t need to create the test database. As mentioned above, this will happen when you run tests.

Project Skeleton

What follows is a bunch of steps: do-this, do-that. At the end of this section you should have a runnable (but empty) project that is ready to start hanging functionality onto.

Then generate a skeleton for your project: django-admin.py startproject YOURPROJECT.

Then dive right in to settings.py and change a few things:

Set the admin email to your email address.
Set the MySQL database according to your web host’s settings.
Timezone.
MEDIA_ROOT, MEDIA_URL and variants — see note below.
TEMPLATE_DIRS — see note below.

To make it easier to use the same settings file to test both locally and on your host, I add the following to my settings.py to set MEDIA_ROOT and TEMPLATE_DIRS:


import os

def full_path_to(path):
    '''This makes this settings file relocatable.'''
    return os.path.join(os.getcwd(), path)

MEDIA_ROOT = full_path_to('static/')
TEMPLATE_DIRS = (
    full_path_to('templates')
)

Then you just have to make sure to cd to the directory where your settings.py lives whenever the app runs (more on this when we deploy to the host). I find this easier than monkeying with PYTHONPATH.

Similarly, the following will change your URLs based on whether you’re running locally or on the host. (Just make sure you test for a directory that only exists locally! My paths are a little different on the host.)


MEDIA_URL = 'http://media.resuman.com/resuman/static/'
if os.path.exists('/home/brian/projects/resuman'):
    MEDIA_URL = 'http://localhost/apache2-default/resuman-static/'

ADMIN_MEDIA_PREFIX = 'http://media.example.com/admin_media/'
if os.path.exists('/home/brian/projects/resuman'):
    ADMIN_MEDIA_PREFIX = 'http://localhost/apache2-default/resuman-admin/'

Now it’s time to generate the app: ./manage.py startapp YOURAPP. Notice that the app name should be different from the project name (otherwise it gets too confusing later on).

Edit your YOURPROJECT/urls.py to include a reference to YOURAPP:


    (r'^funnel/$', include('resuman.jobfunnel.urls')),

Also uncomment the admin lines so you can use the built-in admin app.

Now it’s time to generate the app: ./manage.py startapp YOURAPP.

Now edit YOURAPP/urls.py — paste in the needed bits from the existing urls.py, but drop the admin lines and the include().

In your settings file, add the admin and admindoc apps and “YOURPROJECT.YOURAPP” to INSTALLED_APPS.

Under the YOURPROJECT directory, create a templates directory and a static directory. Copy your HTML files from the mockup to the templates directory. Copy the CSS file to the static directory.

My system is configured by default to serve from /var/www/apache2-default/, and we specified above to fetch static files from http://localhost/resuman-static/, so we need to do the following to make this possible (substituting your paths, of course):


bash$ ln -s /home/brian/projects/resuman/static /var/www/apache2-default/resuman-static
bash$ ln -s /home/brian/projects/django/git/django/contrib/admin/media/ \
    /var/www/apache2-default/resuman-admin

Restart apache.

Change directory to YOURPROJECT and ./manage.py syncdb; ./manage.py runserver.

Browse to http://127.0.0.1:8000/admin/. Log in, and you should see the admin app in all its glory. If the stylesheet didn’t load (ie. it looks really ugly), View Source on the page. At the top, find the URL to that ends something like …/resuman-admin/css/base.css. Copy-paste this into the address bar. It probably doesn’t load. Verify that it is the right URL — if not then change your settings.py to have the right URL base. If the URL is right, then you need to fix your symlink, server config, or permissions (I often get bit by having the wrong permissions).

Adding Some Meat

This wouldn’t be a bad time to push a snapshot into your version control system (e.g. git init; git add .; git commit -m'YOURPROJECT skeleton done').

Now we’re finally ready to write the first view for this app. Edit YOURAPP/urls.py. Lay out the URL map that you want to use for your app. We’re on a tight five day schedule, so don’t go nuts! There’s only time to get a couple of pages done. Don’t worry, you can add more later. Here’s what mine looks like:


from django.conf.urls.defaults import *

urlpatterns = patterns(
    '',
    (r'^$', 'resuman.jobfunnel.views.dashboard'),
    (r'^add/$', 'resuman.jobfunnel.views.add_job'),
    (r'^edit/$', 'resuman.jobfunnel.views.edit_job'),
)

Remember that my toplevel urls.py includes this based on the ^funnel/$ pattern, so each of the patterns above will have …funnel/ as a prefix in the URL.

Now let’s write our first couple of tests. Edit YOURAPP/tests.py and add something similar to the code below. You’ll have to change URLs and logins. The class ViewTestCase is defined in viewtestcase.py, a convenience TestCase subclass I wrote for testing Django views. Copy that file into YOURAPPNAME directory.


from viewtestcase import ViewTestCase
class DashboardViewTestBase(ViewTestCase):
    # Override in subclass to use post/head/etc. Must match a
    # method defined in django.test.TestCase.
    TESTMETHOD = 'get'

    # Override.
    TESTURL = '/funnel/'
    TESTARGS = {}
    TEMPLATE = 'dashboard.html'

class DashboardViewLoginTest(DashboardViewTestBase):
    # We're expecting an error, so set TEMPLATE to None to avoid getting a bogus test failure.
    TEMPLATE = None

    def test_login_required(self):
        """
        Tests that a login is required to view the page.
        """
        self.expect_login_redirect()
        return

class DashboardViewTest(DashboardViewTestBase):
    # This uses TEMPLATE from the parent class.

    # Set username and password and the base class will automagically login the test client.
    USERNAME = 'brian'
    TESTLOGIN = (USERNAME, 'a')

    def test_logged_in_ok(self):
        pass

Now run the test: ./manage.py test. You should see exactly two failures. If a bunch of stuff fails (like built-in django tests), fix whatever is wrong before continuing.


bash$ ./manage.py test
Creating test database...
Creating table django_admin_log
Creating table auth_permission
Creating table auth_group
Creating table auth_user
Creating table auth_message
Creating table django_content_type
Creating table django_session
Creating table django_site
Creating table jobfunnel_job
Installing index for admin.LogEntry model
Installing index for auth.Permission model
Installing index for auth.Message model
Installing index for jobfunnel.Job model
Installing json fixture 'initial_data' from '/home/brian/projects/resuman/../resuman/jobfunnel/fixtures'.
Installed 1 object(s) from 1 fixture(s)
..........EE......
======================================================================
ERROR: test_login_required (resuman.jobfunnel.tests.DashboardViewLoginTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/brian/projects/resuman/../resuman/jobfunnel/viewtestcase.py", line 98, in setUp
    self.fetch_view(self.TESTMETHOD, self.TESTURL, self.TESTARGS)
  File "/home/brian/projects/resuman/../resuman/jobfunnel/viewtestcase.py", line 79, in fetch_view
    self.response = function(testurl, testargs, **extra)
  File "/usr/lib/python2.5/site-packages/django/test/client.py", line 277, in get
    return self.request(**r)
  File "/usr/lib/python2.5/site-packages/django/core/handlers/base.py", line 77, in get_response
    request.path_info)
  File "/usr/lib/python2.5/site-packages/django/core/urlresolvers.py", line 183, in resolve
    sub_match = pattern.resolve(new_path)
  File "/usr/lib/python2.5/site-packages/django/core/urlresolvers.py", line 183, in resolve
    sub_match = pattern.resolve(new_path)
  File "/usr/lib/python2.5/site-packages/django/core/urlresolvers.py", line 124, in resolve
    return self.callback, args, kwargs
  File "/usr/lib/python2.5/site-packages/django/core/urlresolvers.py", line 136, in _get_callback
    raise ViewDoesNotExist, "Tried %s in module %s. Error was: %s" % (func_name, mod_name, str(e))
ViewDoesNotExist: Tried dashboard in module resuman.jobfunnel.views. Error was: 'module' object has no attribute 'dashboard'

======================================================================
ERROR: test_logged_in_ok (resuman.jobfunnel.tests.DashboardViewTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/brian/projects/resuman/../resuman/jobfunnel/viewtestcase.py", line 98, in setUp
    self.fetch_view(self.TESTMETHOD, self.TESTURL, self.TESTARGS)
  File "/home/brian/projects/resuman/../resuman/jobfunnel/viewtestcase.py", line 79, in fetch_view
    self.response = function(testurl, testargs, **extra)
  File "/usr/lib/python2.5/site-packages/django/test/client.py", line 277, in get
    return self.request(**r)
  File "/usr/lib/python2.5/site-packages/django/core/handlers/base.py", line 77, in get_response
    request.path_info)
  File "/usr/lib/python2.5/site-packages/django/core/urlresolvers.py", line 183, in resolve
    sub_match = pattern.resolve(new_path)
  File "/usr/lib/python2.5/site-packages/django/core/urlresolvers.py", line 183, in resolve
    sub_match = pattern.resolve(new_path)
  File "/usr/lib/python2.5/site-packages/django/core/urlresolvers.py", line 124, in resolve
    return self.callback, args, kwargs
  File "/usr/lib/python2.5/site-packages/django/core/urlresolvers.py", line 136, in _get_callback
    raise ViewDoesNotExist, "Tried %s in module %s. Error was: %s" % (func_name, mod_name, str(e))
ViewDoesNotExist: Tried dashboard in module resuman.jobfunnel.views. Error was: 'module' object has no attribute 'dashboard'

----------------------------------------------------------------------
Ran 18 tests in 4.364s

FAILED (errors=2)
Destroying test database...

Let’s write the view so the test will pass. Edit views.py:


@login_required
def dashboard(request):
    return render_to_response('dashboard.html',
                              {'title': 'Dashboard'},
                              context_instance=RequestContext(request))

This view uses a template called dashboard.html. Let’s make that template. Go back to the mockup for the dashboard. Copy everything into templates/base.html, then rip out the content so all you have left is the generic skeleton of a page in your app. Something like this (notice that this is using the title variable passed into the context by the view):


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

<!--
# Resuman: keep track of job applications, cover letters, and resumes
#
# Copyright (c) 2009, Blakita Software LLC
# All rights reserved.
-->

<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
  <meta http-equiv="Content-Type" content="text/html;charset=utf-8" />

  <title>
    {{ title }}
  </title>

  <style type="text/css">
    @import "{{ MEDIA_URL }}base.css";
  </style>

  {% block scripts %}
  {% endblock scripts %}

</head>

<body>

<div id="container">

<div id="menu">
TBD
</div> <!-- end div=menu -->

<div id="header">
<h1>{{ title }}</h1>
</div> <!-- end div=header -->

<div id="content">

{% block content %}
{% endblock content %}

</div> <!-- end div=content -->

<div id="footer">
<div class="nav">
  <ul>
    <li><a href="/">Home</a></li>
    <li><a href="/blog/about">About</a></li>
    <li><a href="/privacy.html">Privacy</a></li>
  </ul>
</div>

<div class="copyright">Copyright © 2009, Blakita Software LLC</div>
</div> <!-- end div=footer -->

</div> <!-- end div=container -->

</body>
</html>

Now take the content you ripped out and put it into dashboard.html:


{% extends "base.html" %}

{% block content %}
<ul id="funnel">
  <li class="phase first">
    <div class="phase">Applied</div>
    <div class="companies">
    <ul class="companies">
        <li class="company">AAA</li>
    </ul>
    </div>
  </li>
  <li class="phase">
    <div class="phase">Confirmed</div>
    <div class="companies">
    <ul class="companies">
        <li class="company">BBB</li>
        <li class="company">CCC</li>
    </ul>
    </div>
  </li>
  <li class="phase">
    <div class="phase">Screen</div>
    <div class="companies">
    <ul class="companies">
        <li class="company">Fubar</li>
    </ul>
    </div>
  </li>
  <li class="phase">
    <div class="phase">Interview</div>
    <div class="companies">
    <ul class="companies">
        <li class="company">Rabuf</li>
    </ul>
    </div>
  </li>
  <li class="phase">
    <div class="phase">Offer</div>
    <div class="companies">
    <ul class="companies">
        <li class="company">Oof Rab</li>
    </ul>
    </div>
  </li>
  <li class="phase last">
    <div class="phase">Start!</div>
    <div class="companies">
    <ul class="companies">
        <li class="company">Rab Oof</li>
    </ul>
    </div>
  </li>
</ul>

{% endblock content %}

Obviously this still just has dummy static data. We’ll get some active data in there very soon, but now we’re ready to rerun the test. It should pass this time. If not, fix the problem. When you get all tests passing, celebrate!

For real data, we need to write the model(s) used by this app. Let’s add another test:


class DashboardViewTest(DashboardViewTestBase):
    USERNAME = 'brian'
    TESTLOGIN = (USERNAME, 'a')

    def test_company_list(self):
        self.expect_div_content('content', 'Foobar Corp')
        return

Edit models.py to add the model:


from django.contrib.auth.models import User
from django.db import models

class Job(models.Model):
    applicant = models.ForeignKey(User)
    company = models.CharField(max_length=80)

The test depends on having a job in the database. Let’s set up a test fixture. Edit YOURAPPNAME/fixtures/initial_data.json:


[{"pk": 1, "model": "auth.user", "fields":
    {"username": "brian", "first_name": "", "last_name": "",
     "is_active": 1, "is_superuser": 0, "is_staff": 0,
     "last_login": "2009-02-06 13:50:02", "groups": [], "user_permissions": [],
     "password": "sha1$0fcc8$c4cf5184f5c005f90165e782fb090e7d75b72986",
     "email": "brian@example.com", "date_joined": "2009-02-06 13:44:04"}},
 {"pk": 2, "model": "auth.user", "fields":
    {"username": "alan", "first_name": "", "last_name": "",
     "is_active": 1, "is_superuser": 0, "is_staff": 0,
     "last_login": "2009-02-06 13:50:02", "groups": [], "user_permissions": [],
     "password": "sha1$0fcc8$c4cf5184f5c005f90165e782fb090e7d75b72986",
     "email": "alan@example.org", "date_joined": "2009-02-06 13:44:04"}},
 {"pk": 1, "model": "jobfunnel.job", "fields":
    {"position_url": "http://example.com/career/", "title": "Foobar Eng",
     "company_url": "http://example.com/", "company": "Foobar Corp", "applicant": 1,
     "phase": "Apply", "date": "2009-02-11", "position": "Engineer",
     "notes": "Applied via website"}}
]

This will populate the database with a couple of users, both with password “a”, and a job before each test runs.

Run this test, expecting exactly one failure — the content div does not contain the expected string. Let’s grab the list of jobs in the view and pass it into the template:


@login_required
def dashboard(request):
    jobs = models.Job.objects.all()
    return render_to_response('dashboard.html',
                              {'title': 'Dashboard',
                               'jobs': jobs,
                               },
                              context_instance=RequestContext(request))

I won’t paste all of the code here again, but we need to edit the template to use the jobs list:


    <ul class="companies">
      {% for job in jobs %}
        <li class="company">{{ job.company }}</li>
      {% empty %}
        <li class="company">No jobs in this phase.</li>
      {% endfor %}
    </ul>

Now rerun the test and expect it to pass. Hooray!

One last refinement before we quit for today: users shouldn’t be able to see each others applications. The way this is coded, all jobs are going to show up on everybody’s dashboards. Not good. Here’s another test that checks that the application for Foobar Corp only shows up on Brian’s dashboard, not on Alan’s.


class PrivateDashboardViewTest(DashboardViewTestBase):
    USERNAME = 'alan'
    TESTLOGIN = (USERNAME, 'a')

    def test_private_applications(self):
        assert('Foobar' not in self.get_div_content('content'))
        return

Run the test, watch it fail, and then change one line in the view:


    jobs = models.Job.objects.filter(applicant=request.user)

Now the all the tests should pass.

Push a copy of your code into your version control tool. Take a break, you deserve it.

For “homework”, put together your other views in the same way as this one. We’ll look at deployment tomorrow.

Five Days to a Django Web App: Day Two, Mockups Welcome back. Ready for Day...
Five Days to a Django Web App: Day One, Get Ready This is the first in...
Five Days to a Django Web App: Day Four, Deployment Thanks for your patience, and...

Five Days to a Django Web App: Day Two, Mockups

2009-02-10T19:01:42Z

Welcome back.

Ready for Day Two? Did you get your “hello world” app running on your host?

Where We Stand

Yesterday we nailed down our concept, bought a domain name and hosting, set up our toolkit, and deployed a practice app on the host.

Here’s what I’m going to build: a web app to keep track of job applications, cover letters, and resumes. The working name for the project is “resuman”. Unfortunately, that obvious domain name was already registered and I’m not up for buying from the current owner. So after trying a bunch of more-or-less obvious combinations, I grabbed “yresu.me”, and I’ll deploy the app at “trackm.yresu.me”.

Notes on Domain Registration

It’s worth making a couple of notes here on the domain registration process. I use GoDaddy for my registrations. In the past I’ve done registration with my preferred host, but if you end up switching hosts it’s a hassle to move the domain registrations. Another complication: hosting services like DreamHost only support registrations on the major TLDs (.com, .org, .net, .info), while a full-service registrar like GoDaddy supports alternatives like .me.

Lastly, I like the bulk domain search service from Dotster. You can search for up to 50 names at a time. So if your first choice is unavailable, use Dotster’s bulk search tool to enter a whole mess of variations and see what’s available. This is the process I used to find yresu.me — the search included resu.me, esu.me, su.me, and myresu.me, all of which were taken.

Make Some Mockups

Before we commit any design ideas to HTML, let’s take some great advice from 37signals: sketch a bunch of drafts on paper first. To back up this idea, check out this story posted by Jeff Atwood about quantity vs. quality:

The ceramics teacher announced on opening day that he was dividing the class into two groups. All those on the left side of the studio, he said, would be graded solely on the quantity of work they produced, all those on the right solely on its quality. His procedure was simple: on the final day of class he would bring in his bathroom scales and weigh the work of the “quantity” group: fifty pound of pots rated an “A”, forty pounds a “B”, and so on. Those being graded on “quality”, however, needed to produce only one pot – albeit a perfect one – to get an “A”.

Well, came grading time and a curious fact emerged: the works of highest quality were all produced by the group being graded for quantity. It seems that while the “quantity” group was busily churning out piles of work – and learning from their mistakes – the “quality” group had sat theorizing about perfection, and in the end had little more to show for their efforts than grandiose theories and a pile of dead clay.

So my advice to you is to produce no fewer than five different designs on paper. It only takes a few minutes to sketch something out and see what it looks like. I did a handful of drawings on paper with my kids’ crayons, and then some refinements on my whiteboard once I had chosen the general design. So use whatever media you have handy, and don’t worry about getting it perfect. Run your sketches by a couple of innocent bystanders for some feedback.

Once you’ve got a good design on paper, make mockups in HTML and CSS. Steve Dennis at subcide.com has a great walkthrough on creating a CSS layout from scratch. If you don’t have an established process for building up a design, follow his tutorial. Putting together the HTML+CSS takes a little more time than sketches on paper, but it’s still worth doing a couple of different designs to see what grabs you. I worked up two different CSS layouts on top of the same HTML. Run these designs by some people — now that you’ve got some code, you can upload it to your site and email a link to some friends. Pick the better of the two (or three).

Better yet, upload the mockups to your host and leave a comment here with a link to them. Traffic here is low enough that I should be able to reply with feedback. (Although feel free to circulate a link to this series to a hundred of your friends, or post it to digg. I’ll at least reply to the first 50 comments…)

Tomorrow we’ll start writing some code.

Five Days to a Django Web App: Day One, Get Ready This is the first in...
Five Days to a Django Web App: Day Four, Deployment Thanks for your patience, and...
Five Days to a Django Web App: Day Three, Coding Thanks for coming back for...