The Problems

Task Confusion

If you work on lots of similar tasks, especially if they are sometimes yanked out from under you while you work on them and replaced with other similar tasks, they start to run together. You remember that you just fixed a timeout issue that was reported by – who was that, that you worked on this morning? The Flarg Shoppe project? No, you did that yesterday. But the timeout fix was certainly on the Quux module, and only Flarg and Butterford use that module. Wait…

Splattered Attention Span

Your brain can only handle so much task switching. As you task switch more, each context transition becomes more fragile, and you can have a “bad switch”. For me, a “bad switch” results in a complete loss of context – a short term memory wipe where you have to reboot and rebuild your current memory state.

Morale Shutdown

We know that task switching has negative affects on short-term memory. We also know that we do not accomplish more by trying to do multiple things at once. It might be difficult to convince management of this, but you might just get through. But sometimes, management finds this is necessary for political reasons anyway. (It actually serves a purpose for management – they can point to their actions and show that the failure doesn’t stem from the Socially Worst Sin Of Management – Inactivity In The Face Of A Problem. It is actually much more difficult to defend against an accusation of inactivity than an accusation of failure. But don’t get all high-and-mighty when you realize this – it’s the same reason we coders pull extra hours on a late and doomed project.)

But there is a point – for me, usually by the fourth activity yanked out from under me in one day, preventing me from accomplishing anything because everything is important – where Complete Morale Shutdown occurs. My brain has checked out, turn up the headphones, open Eclipse. Don’t forget to scroll the code window every few minutes until 5PM.

Stress Body Buzz

This was the worst. I’ve only had this maybe three times, and I get very worried that this has a big negative affect on health. I’ve walked out of the office literally unable to calm down. On the worst occasion, I called my doctor on Sunday because I still hadn’t recovered from Friday.

I’ve found that an extremely rigorous bike ride can greatly speed recovery.

The Solutions

Work on the Big Problem

The first lesson, of course, is that frequent task-switchng doesn’t really improve throughput or profitability – and most certainly destroys it – so seek to remove the fundamental underlying issues. This, of course, takes time and management involvement and systems thinking, and there probably aren’t any standard solutions to this problem.

The rest of this article is about interim fixes that you will need while you work on the big problem.

Turn Off E-Mail Alerts

Do you really think you’re going to forget to check your e-mail? Really?

If you can arrange it, steal time by closing e-mail and telling your manager to physically get you if something really requires immediate attention.

Use Tactile Reminders

If you work on multiple tasks in one day and they are similar, it is very easy to become confused which task for which customer you are working on. Tactile reminders such as Post-It(r) notes or index cards that you move around are very useful for this purpose. They are invaluable for rebuilding memory state during a Splattered Attention Span reboot.

Make TDD Fail

TDD in any form is a big help when coding. In a normal environment it adds a rhythm which keeps momentum from leaving, but in a frenetic environment it keeps you on task. Can’t remember what you were doing before that interruption? Click the “run test” button… ah, that was it.

If you have a research phase to your current task, write a failing test immediately for any requirement discovered. This is a bit different from the normal TDD rules, where you don’t write the next failing test until you’ve passed the last test. In this mode, you collect failing tests when necessary, and commit when they reach zero. Note that zero can still be reached a couple times on one task.

Unless you can just spit out the test’s code in one go, simply make the test fail with the fail() command and a very descriptive test name; otherwise, you are inviting Splattered Attention Span.

Use the Shell’s “&&”

As you thrash your short-term memory, your idea of what is a “long running” command becomes shorter and shorter. Longer than 30 seconds is hard to keep focused. If you need to run multiple steps, you can chain them together with your shell’s “&&” operator. Joining multiple steps reduces opportunity for a context-switch-overload full-stop.

Learn VCS Hygiene

You should become adept at using your version control system from the command-line to inqure about what changes exist. Make it a strong habit to check before coding something new and review all changes before commit. It’s harder to get an overview when you are frequently task-switching.

Use git

Git is wonderful, and it even makes a good SVN client (once you bootstrap it from your SVN repository – which is admittedly painful). Become familiar with with “git stash” – a task switcher’s savior.

Switching between branches (when necessary – running multiple branches is a bad practice when it can be avoided) is also much less painful to the short-term-memory impaired in git than SVN. In git, it is a practically instantaneous operation; in Subversion, it is an assured short-term memory snow crash.

Automate, Automate, Automate

There are many technical investments that you can to reduce the affect of task switching. Multi-step manual processes are much more costly in a stressful environment than other environments, because each step transition invites a high probability of attention-span splatter. Consequently, investing in automation is a big win.

Of course, you are trying to invest in automation in a frenetic environment… how could this work? The answer is that you need to grow these automations from tiny improvements. A few notes I’ve learned along the way:

• For the most part, you don’t need TDD if it is a process that will be run only by you or other developers and it prints a stack trace on failure.
• A small batch file that does three sequential things is a big win for a small investment. Use a language that you won’t mind maintaining when it grows into a big, hairy ball.
• Weigh the possibility of reuse of your application code in your automations. If there is potential, either use the same language as your application code or a scripting language which you can bind to it (e.g. Python/Jython, Ruby/JRuby).

Boost often provides friend classes and template friends to do this, but this can be a lot of work. What I’ve done is made the member protected, then:

struct TestableGuy : public Guy {
    inline State& state() { return state_; }
};

In my testing code:

      Guy g(Point(99,99), ts);
      reinterpret_cast<TestableGuy&>(g).state() = s;

I’m using reinterpret_cast<> to avoid needing to maintain constructors on TestableGuy as I add them or change them on Guy. Since no new data members or virtual functions were added, we can be assured that the classes have the same binary representation.

QuickCheck is the gold standard for testing pure functional (side-effect free) code in Haskell. It allows you to assert properties of functions, such as “this function returns a non-negative result for any input.” QuickCheck generates input cases and checks the proposition automatically.

HUnit is the usual xUnit clone for Haskell. It deals with actions (Haskell’s term for functions having side-effects). You perform some action then assert the resulting state has been correctly mutated.

It is mighty convenient to use both for TDD on any non-trivial program.

Test Collection

Relying on the coder to remember to add tests to suites will eventually produce orphaned suites. Then comes the false confidence: I ran all the tests and they passed, and I know it doesn’t break assertion X, because there’s a test for that. I’ve seen it.

As OneSock has gotten a little bit bigger (it’s still small, don’t worry), I start to worry about this. I did a bit of research and found some test-collection tools:

quickcheck-script

This script will scan the source files supplied on the command-line for QuickCheck assertions and run them. It is very useful and easy to use. The drawback here is that it will not run my HUnit tests.

hstest

This program will scan for QuickCheck1 and HUnit tests in all sources in the current directory and run them. The drawback is its current lack of support for QuickCheck2. The philosophy is right for what I want to do – have a simple “brain idle” command for my TDD cycle that runs all the tests.

test-framework

This package provides a class with instances for HUnit, QuickCheck1, and QuickCheck2. For the most part, manual collection is required; however, there is a test-framework-th package which uses the TemplateHaskell extension to provide a meta-function that scans the current source file for tests and collects them.

The Decision

I’ve decided to go with hstest.

quickcheck-script doesn’t run HUnit tests.

While test-framework seems to have more mindshare and also seems to be well-thought-out, there is no script to automatically collect tests. The TemplateHaskell trick is nice; however, introducing another layer of complexity just for test collection is a bit overwhelming at this early stage—especially because it doesn’t solve the original problem: now a programmer must remember to register all source files rather than all tests in a test-runner program.

To work around the current lack of support for QuickCheck2, you can supply parameters to hide and expose packages. I’ve written this shell function:

function t() {
	hstest --expose-package=QuickCheck-1.2.0.0 --hide-package=QuickCheck-2.1.0.3
	rm -f hugsin
	return $?
}

I’ve found that my current few properties run unaltered under QuickCheck1.

I’ve also specified QuickCheck1 in my cabal file for the moment so that I don’t use features that hstest doesn’t like. This provokes a warning:

jfelice@flarp64:~/src/onesock$ cabal configure
Resolving dependencies...
Configuring OneSock-0.1...
Warning: This package indirectly depends on multiple versions of the same
package. This is highly likely to cause a compile failure.
package OneSock-0.1 requires QuickCheck-1.2.0.0
package Crypto-4.2.1 requires QuickCheck-2.1.0.3
jfelice@flarp64:~/src/onesock$

No build issues occur. Maybe the linker is eliminating all QuickCheck references (which are unreachable from main).

  for (int i = 0; i < data.length; ++i) {
    if (i%16==0) System.err.println();
    System.err.printf("%02x ",data[i]);
  }
  System.err.println();

The C and C++ equivalents are almost identical.

1. Pair up.
2. One test is written first.
3. Only as much functionality as is needed to make the test pass is written.
4. The next test is written…

…and so forth.

We moved through parsing the command-line easily, then coded the test for displaying a single digit. We noticed we were at a particular sort of impasse:

Implementing as little code as possible to pass this test could not produce code which could survive the next test. At least, not in a way that I can see.

What should you do? Should you write the code, debug it and pass the test, then add the next test and delete the code? This seems like an odd question to ask. The only reason I even ask it now is I’ve found some people who really advocate this.

Another example: I listened during the Roman Numerals kata as one team made first I, then II and III pass by coding:

  def convert(n)
    n.split("").size
  end

If I understand proponents’ arguments, in writing the code you are learning, and this learning will dominate the inefficiencies in this process over time.

I honestly think this is a misreading of agile. Agile was built in the context of an industry addicted to wasteful, heavy and rigid process. Should TDD become a wasteful, heavy and rigid process?

How do I proceed in these cases? First, I have nothing against writing more than one test up front. If I can think of several of the corner cases and potential pitfalls right now, I will write them down in xUnit. Sometimes switching from researching the requirements to coding is an expensive operation, and in these cases, I’ll lay out five or ten tests documenting the requirements. Sometimes, I have no idea how I’m going to implement something. Writing a batch of tests up front allows me to explore the corner cases and think more about designing a useful approach.

Given the multiple tests, I have no problem writing the more complicated algorithm first. We’ve already determined that we can’t (or perhaps just don’t know how) to evolve it, so we’re going to end up going for it anyway.

    public void testCallsRollbackIfSendingAuthResponseFails() throws Exception {
        final MockSender sender = new MockSender();
        final MockSession session = new MockSession() {
            @Override
            public DASAuthObj authorize(final DASAuthObj authObj) {
                mgr.dispatch( ... // AGHAGHAGH!
                return super.authorize(authObj);
            }
        };
        final DASSessionManager mgr = new DASSessionManager(session, sender);
        ...

[EDIT: OK, you can do it manually with Java arrays. Still icky.]

    public void testCallsRollbackIfSendingAuthResponseFails() throws Exception {
        final MockSender sender = new MockSender();
        final DASSessionManager[] mgrHolder = new DASSessionManager[1];
        final MockSession session = new MockSession() {
            @Override
            public DASAuthObj authorize(final DASAuthObj authObj) {
                mgrHolder[0].dispatch(new DASCloseAuthSinkPacket(1, true, 42));
                return super.authorize(authObj);
            }
        };
        final DASSessionManager mgr = new DASSessionManager(session, sender);
        mgrHolder[0] = mgr;

The utilities expect data to be stored in a data.frame. The first column must be named “Date” and have “Date” class. The second column is currently ignored; however, the intention is that it indicates the ticket category (usually represented on the board by ticket color), and so it should be a factor. Columns three onward (as many as you have flow steps) are tallies of each work flow step for the day, in reverse chronological order – starting with “Done” and ending with “Backlog”, in other words.

The easiest way to get this data into R is to store it in a CSV file and use read.csv to load it. Here is some sample data:

Date,Category,Done,QA,Development,Ready,Backlog
2009/10/20,Project,0,2,2,0,0
2009/10/20,Urgent,0,0,1,0,0
2009/10/22,Project,1,2,1,0,0
2009/10/22,Urgent,0,0,2,0,0
2009/10/23,Project,1,2,2,0,0
2009/10/23,Urgent,0,0,2,0,0

Load this into R with:

> my.data <- read.csv("my_data.csv")

An alternate method for maintaining your data within R is to use the fix() function on a daily basis, which is what I do. Two warnings, however:

1. Make sure to save your workspace so that you don't lose your stats.

2. fix() will strip the "Date" class from the Date column.

To re-add the Date class after using fix(), do this:

> my.data$Date <- as.Date(my.data$Date)

Plotting your CFD

Check out the rkanban sources into the "rkanban" directory under your R working directory and source the kanban.R file.

You can then plot your CFD from within R like so:

> source("rkanban/kanban.R")
> PlotCFD(my.data)

Guice in Ten Seconds

You annotate your classes types’ primary constructors, create “modules” which bind concrete types or singleton instances to interface and abstract types, then ask Guice to create you one of some type. Guice will automatically create all dependencies (constructor parameters) for that type before invoking the primary constructor, and if any of those types require dependencies create those first, and so forth. For example, if your class Foo needs some kind of PackagingStrategy (presumably an interface) passed to its constructor, and your module binds PackagingStrategy.class to BinaryPackagingStrategy.class, then when you ask Guice for an instance of Foo, it will create a new BinaryPackagingStrategy, “newing up” its constructor parameters recursively, and pass this to Foo’s constructor.

The benefits of this scheme are in the class of “hard to describe, but definitely there.” First and foremost, it improves system design, making interfaces clearer and more self-documenting by removing hidden dependencies on things like Singletons and Monostates. This improves testability because mucking with the global state encapsulated in Singleton and Monostate patterns requires a much deeper knowledge of classes under test and some often-shady practices. Without a good DI framework, passing a dependency down several layers of object graph is complicated and provides extra coupling, making the Singleton pattern much more attractive. With the DI framework, there’s really no inclination to make the dependency of a class anything other than a constructor parameter.

But, More Boilerplate?

There’s plenty of boilerplate that Guice removes, mostly in the “bean wiring” category, and this is good. Interestingly, there are areas in which I’ve found myself writing more boilerplate with Guice, and an instance of this is what I’d like to discuss today.

I work with some people who have become thoroughly disgusted with OOP and advocate for functional style with immutable data types. I appreciate functional style with immutable data types, but I must say that the Strategy pattern is something OOP does well and not something that FP does nearly as well. I tend to use strategy pattern quite a bit in our XPay (and in our C++ product, DAS). One reason to prefer Strategy pattern to switch statements is to have a single point of control over which strategy implementation to use. This use case usually conjures itself into being in a Factory method pattern which has the single switch statement which provides an instance of the concrete type based on input parameters.

In Guice, you can inject a Provider<Foo> and Guice will automatically create a type that produces instances of Foo. This is useful for dependencies. So our factory ends up looking something like this:

class FooFactory {
 
    private final Provider<TypeAFoo> typeAFooProvider;
    private final Provider<TypeBFoo> typeBFooProvider;
    private final Provider<TypeCAFoo> typeCAFooProvider;
    private final Provider<TypeCBFoo> typeCBFooProvider;
    private final Provider<TypeDFoo> typeDFooProvider;
    private final Provider<TypeEFoo> typeEFooProvider;
 
    @Inject
    public FooFactory(
            final Provider<TypeAFoo> typeAFooProvider,
            final Provider<TypeBFoo> typeBFooProvider,
            final Provider<TypeCAFoo> typeCAFooProvider,
            final Provider<TypeCBFoo> typeCBFooProvider,
            final Provider<TypeDFoo> typeDFooProvider,
            final Provider<TypeEFoo> typeEFooProvider
            )
    {
        this.typeAFooProvider = typeAFooProvider;
        this.typeBFooProvider = typeBFooProvider;
        this.typeCAFooProvider = typeCAFooProvider;
        this.typeCBFooProvider = typeCBFooProvider;
        this.typeDFooProvider = typeDFooProvider;
        this.typeEFooProvider = typeEFooProvider;
    }
 
    public Foo get(char type, boolean inverted) {
        switch (type) {
        case 'A':
            return typeAFooProvider.get();         
        case 'B':
            return typeBFooProvider.get();         
        case 'C':
            if (inverted)
                return typeCAFooProvider.get();
            else
                return typeCBFooProvider.get();
        case 'D':
            return typeBFooProvider.get();         
        case 'E':
            return typeBFooProvider.get();         
        default:
            throw new FooFactoryException("Can't determine type.");
        }
    }
};

Ugh!

This is “better” boilerplate than the bean wiring before in that it hints that we might be able to concoct a general solution; however, I haven’t yet found it. One solution which I’ve rejected is the “injecting an injector” solution:

class FooFactory {
 
    private final Injector injector;
 
    @Inject
    public FooFactory(final Injector injector) {
         this.injector = injector;
    }
    public Foo get(char type, boolean inverted) {
        switch (type) {
        case 'A':
            return injector.createInstance(TypeAFoo.class);
        case 'B':
            return injector.createInstance(TypeBFoo.class);
        case 'C':
            if (inverted)
                return injector.createInstance(TypeCAFoo.class);
            else
                return injector.createInstance(TypeCBFoo.class);
        case 'D':
            return injector.createInstance(TypeDFoo.class);
        case 'E':
            return injector.createInstance(TypeEFoo.class);
        default:
            throw new FooFactoryException("Can't determine type.");
        }
    }
};

The reason I’ve rejected this approach is that it prevents Guice from checking the entire dependency graph at boot – Guice doesn’t know which types you are going to create with the injector, and this has to defeat a lot of it’s validation magic.

.\Components/CommComponents/FDMSInterleaveTcpipComm/Test_FDMSInterleaveTcpipComm.h(106) : fatal error C1001: INTERNAL COMPILER ERROR
        (compiler file 'msc1.cpp', line 1794)
         Please choose the Technical Support command on the Visual C++
         Help menu, or open the Technical Support help file for more information

was caused by:

    template<typename VersionPolicy> class SVDotPacketProductionPolicy;
    class SVDotPacketProductionPolicy<SV24VersionPolicy>;
    friend class SVDotPacketProductionPolicy<SV24VersionPolicy>;
    class SVDotPacketProductionPolicy<SV40VersionPolicy>;
    friend class SVDotPacketProductionPolicy<SV40VersionPolicy>;

Include\boost/bind/arg.hpp(25) : fatal error C1001: INTERNAL COMPILER ERROR
        (compiler file 'msc1.cpp', line 1794)
         Please choose the Technical Support command on the Visual C++
         Help menu, or open the Technical Support help file for more information

This one started after I added

#include <boost/bind.hpp>

to a file and a simple, every-day use of boost::bind(). Moving the #include directive up before a number of other standard includes resolved the issue.

C:\projects\DASV2>scons
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
cl /FoComponents\AuthComponents\Credit\CCMps\Release\Tests.obj /c Components\AuthComponents\Credit\CCMps\Release\Tests.cpp /nologo /
TP /nologo /MT /W2 /GX /GR /O2 /Zi /Zm200 /D "WIN32" /D "_WINDOWS" /D "NDEBUG" /D "_MBCS" /FD /D "_WIN32_WINNT=0x0400" /D "_WIN32_DC
OM" /IWindowsSDK\Include /IInclude /Icryptlib /IXML\Xerces\src /I. /IComponents\AuthComponents\Credit\CCMps
Tests.cpp
Include\boost/function/function_template.hpp(514) : fatal error C1001: INTERNAL COMPILER ERROR
        (compiler file 'msc1.cpp', line 1794)
         Please choose the Technical Support command on the Visual C++
         Help menu, or open the Technical Support help file for more information
scons: *** [Components\AuthComponents\Credit\CCMps\Release\Tests.obj] Error 2
scons: building terminated because of errors.

06/01/2009 – This one was the result of passing a pointer to a static member function to boost::function1<>.

C:\projects\DASV2>scons
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
python.exe Utilities/CxxTest/cxxtestgen.py --runner=XmlPrinter --have-eh --have-std --check-memory -o CBase\ISOMsg\Debug\Tests.cpp CBase\ISOMsg\Test_Fields.h
cl /FoCBase\ISOMsg\Debug\Tests.obj /c CBase\ISOMsg\Debug\Tests.cpp /nologo /TP /nologo /MTd /Od /W2 /GX /GR /Zi /Zm200 /D "WIN32" /D "_WINDOWS" /D "_MBCS" /FD /D "_WIN32_WINNT=0x0400" /D "_WIN32_DCOM" /D "DEBUG" /D "_DEBUG" /IWindowsSDK\Include /IInclude /Icryptlib /IXML\Xerces\src /I. /ICBase\ISOMsg
Tests.cpp
.\CBase/ISOMsg/Test_Fields.h(157) : fatal error C1001: INTERNAL COMPILER ERROR
        (compiler file 'msc1.cpp', line 1794)
         Please choose the Technical Support command on the Visual C++
         Help menu, or open the Technical Support help file for more information
scons: *** [CBase\ISOMsg\Debug\Tests.obj] Error 2
scons: building terminated because of errors.

06/18/2009 – This one is the result of attempting to use a template member of template class. Here’s the (abbreviated) code:

template<typename C, C v>
struct ThreeArgConstructor
{
    template<class BaseT>
    struct Type : public BaseT
    {
        Type(int length)
            : BaseT(length, "TEST FIELD PACKAGER", v)
        {}
    };
};
 
template<
      BaseT
    , ConstructorT
    >
class Foo
{
    typedef typename ConstructorT::template Type<BaseT> TestPackager;
 
    void run() {
        TestPackager(10);
    }
};
 
int main() {
    Foo<Bar, ThreeArgConstructor<bool, false> >().run();
    return 0;
}

A significant portion of your activities aren’t visible to your management, possibly because

• you have become the “product expert” which consults with others or
• your products’ production deployments require significant developer attention

Therefore,

Divide your whiteboard up into columns representing your work flow (e.g. “Backlog,” “Ready”, “Develop”, “QA”, “Done”) and obtain a large supply of Post-It(tm) notes. Use separate colors for the items assigned to you by your management and the items which come to you as the domain expert. If necessary, track enough information to build a cumulative flow diagram.

Discussion

There is something visceral about this approach that conveys understanding in a way which is not possible by explanation or discussion. It may be the case that your management knows what is happening but doesn’t quite “get it.”

This is a minimum useful subset of software kanban and an instance of “Information Radiator.”

A cumulative flow diagram can be created by simply counting the number of tickets in each bucket each morning and entering these into a spreadsheet in backwards order. You can read your average work-in-process (WIP) and cycle time from this diagram, making it an incredibly inexpensive way to measure the amount of effort spent on visible work versus other work.

Observed Instances

One

[to be filled in]