I believe in TDD. Test driving code clarifies the task at hand, and provides continuous confirmation that your code works in small parts. It allows you to refactor with confidence, and reshape your codebase throughout it’s life.

I believe in Continuous Integration. This implies a continuous integration server that run tests against integrated code. This also means avoiding patterns of reduced integration with other people, such as long lived code branches.

I believe in Pair Programming. The benefits are substantial. Pairing results in better developers, a better codebase, better thought out solutions, higher productivity, and a real team of people.

I believe in Empowered Teams. People who function as cogs in the machine cannot contribute to their fullest. Build a team, make them capable of their own decisions, support them, and let them perform.

I believe in Frequent Releases. Frequent releases results in smoother releases, and a greater dynamism to market or client needs.

I believe in being a General Technologist. Developers should not over-specialize to the point that they can’t deliver software without several other specialists. It’s better for most team members to understand and be proficient at in all areas of software delivery, with a few specialists being an added bonus to the quality and direction of the product.

I believe in Simplicity. Excessive complexity is too common in software development. While some complexity is unavoidable, I favor simpler solutions, simpler tools, and simpler products.

Above all, I believe in high quality, well crafted software.

I can’t believe I lost decisively at all.

Being way behind for the entire competition was an indicator of having a poor chance, but I after wishing really, really hard to win, it was somewhat of a surprise.

Seriously, this was the most fun programming I’ve had in a long time, and I look forward to the next competition. I want to say congratulations to Gábor Melis, who’s bot, bocsimacko, took first place. I remember when I first saw his bot on the Dhartmei server (an alternate TCP server we could use to get more frequent games); it had a win rate of 98%. That was playing the best bots at the time such as dmj11 and McLeopold. I spent a lot of time trying to watch his bot and think about how he made it. Now that the source code is available here. I plan and playing with it and understanding it better. His win I think has clinched my decision to learn lisp, and use that language in the next competition.

The high point of the contest was when I reached 13th place, and spent a good part of a week in the top 30′s. 2 or 3 weeks before the end of the tournament, I had been ranked in the 50′s and 60′s. Then the map generator changed and it slipped down to about 150. Alas, I was not able to improve my bot much during those 2 weeks. I had really hoped I would be in the top 100, and for the first day and a half of the finals, I was in the 80′s. It made it an exciting end to the contest. In the end, my rank was 122 (out of 4617).

The other major unfortunate occurrence was that I attempted to rewrite my bot to include better modeling of future states of the board, and to allow for a multiplanet coordinated attack system. That rewrite never performed better than the pre-rewrite version of my bot, so I never got to use that code. The next phase of my plan was to begin simulating out the board with an enemy AI and select the most favorable future based on that simulation. However, my dreams were bigger than my time available, and was never realized.

If you would like to see the final submission version of my bot, it’s located here:

If you would like to look at the rewrite version, it’s located here

Approach

I made the somewhat regrettable decision to use Java for the submission. This was mostly due to my comfort with the language, and my fear of hitting a performance bottleneck later when I wouldn’t have time to switch languages. I say that it’s somewhat regrettable because I did eventually hit performance issues indicating some languages would perhaps have been too restrictive. However, Java is a verbose and clunky language, and I found it quite painful to use (I’ve been programming in ruby the last 2 years).

I wrote some scripting do all the simple tasks that I needed such as compiling and running my code against all the example bots, but a coworker of mine, Tony Pitluga, provided some better scripting so I used that.

My development approach was fairly modest. Just add simple actions that improve my rating. Once I had all the core actions done, tweak them to make them better. Test against previous versions of my bots and the example bots. Eventually, when I had a fairly good bot I was going to learn and then implement AI algorithms such as A* or minimax.

Implementation

In the end, my general algorithm was:

1. Rescue my planets that will be lost, picking the closest available planets

2. Try to do a simple snipe on the enemy if the oppurtunity presents itself

3. Attack the most valuable planets as determined by a simple value function, using my closest planets, and only doing single (source) planet attack.

4. Move remaining ships into the nearest, most valuable neutral that I don’t think is at risk of being taken by the enemy, unless the enemy or I are attacking each other

5. Move remaining ships into my most strategic planets, where the most strategic planets those being closest to the enemy (a combination of min and average distance)

Along the way there were a few major things that made a big difference in my elo score.

1. When attacking, I added logic to not consider neutrals if the return on investment was better with other neutrals by waiting some number of turns.

2. The logic to blindly stream planets into close neutrals was implemented at the same time, and seemed to work well combined with the previous point

3. Deciding whether to continue aquiring planets. For instance, if my growth rate is better than the opponent, then only consider attacking the enemy.

4. Playing out the future states of individual planets. That is, playing out all fleets in motion and then considering the future state when making certain decisions. Most notable was determining my most strategic planets, and figuring the ships needed to rescue and win a planet.

In addition to those significant additions, there were a number of important bits of logic that needed to be considered. One of the most important was deciding how many ships to send away from a planet. The simple solution to that problem is to look at the closest enemy planet, and not send so many that that enemy planet could take your planet. While it had obvious shortcomings that resulted in a lot of losses, other simple variations that I tried were worse, often making my bot too defensive.

I believed that the best solution to that decision is to find the difference in the ships that could attack a planet versus rescue it, and then base my decisions on that. I attempted this in my rewrite that I did, but it did not seem to have an appreciable improvement. I’m not sure why, but that was when I started running out of time, so I didn’t investigate this further.

Testing

Initially, I attempted to test my code using scenarios. ie: Given 2 neutrals at the same distance, I’ll take the one with the better growth rate. This quickly became an annoyance to me, as they were constantly breaking (and being invalidated) when I added more complicated decisions, and I would have to keep adding to each one to reflect that complexity. For instance, if I now value neutrals differently based on proximity to the enemy, I would have to change all my scenarios. I deleted all those tests after the first week. I felt that that style of testing did not help me to make my algorithms more correct the first time, or really keep them correct.

In the end, the tests that were most valuable were the ones that tested calculations. For instance, trying to determine the ships needed to win a planet based on incoming fleets at varying distances. The nature, or “truth” of these kinds of assertions never fundamentally changes, is difficult to get right the first time through manual testing, and is greatly benefitted from having tests when the code is changed (such as when I optimized it). In particular, that logic can be wrong and your bot could do something close to correct a lot of the time, masking the error.

My regression testing was to test my changed bot against my previous versions and the example bots and note the difference in wins and losses. I would then look at why I lost on specific maps, and tweaked code to address what I saw. I would then rerun those maps against multiple bots to see if I fixed or improved that scenario.

Outside of that, testing on the Dhartmei server was my last major prerelease test, though, it suffered from variability due to the fluctuating set of bots on the list, so it’s difficult to compare your new bots performance without running multiple versions on the server for a while.

Lessons Learned

1. Keep an eye on the forums. I didn’t realize the map generator had changed until days after it happened.

2. Leverage Dharmei’s server more. I spent a lot of time doing regression testing against old versions of my bots, but that isn’t the best metric. The version of my bot that I ended up submitting surprised me; it was only marginally better against it’s previous versions, but it’s elo on the Dhartmei server jumped 200 points over the previous version.

3. Don’t take one computer as the final word when evaluating performance. I had despair towards the end when I found my simple simulations took 400 milliseconds on my 5 year old desktop. When I ran it on my 3 year old MacBook pro, it ran in less than 80 milliseconds. For all I know, it could run in less than 10 milliseconds on newer hardware.

4. Know more than one high performance language, and make sure you like one of them

5. Set some serious time aside closer to the end of the tournament. A couple of hours of work here and there while holding a baby won’t allow you solve hard problems.

6. Get some hardware. Perhaps I’ll pay for some amazon ec2 instances to run simluations with. They’ll run faster, and I may be able to parallelize the testing of various changes.

Error 51: Unable to communicate with the VPN subsystem

First, I read that I could do this in the terminal:

Since I don’t have a CiscoVPN directory under startup items, that didn’t help me. Then I read that you need to to repair disk permissions using disk utility. I did that and it found several inconsistencies. Good stuff, though, it also did not solve my problem.

Then, I read that I could do this:

The command didn’t produce any output, nor did it fix my problem.

Finally, I came across a site that told me to do this:

This finally fixed my problem. Here is the sequence of commands that I ran and the output:

Hopefully, this will save someone some time.

Almost 3 years later, I got to help facilitate the game for the Agile Atlanta Users Group. Only this time, I got to be one of the customers. It was a blast.

In case you aren’t familiar with the Lego game, the objective is to form a team of people and have them go through 3 agile iterations developing a lego creature via story cards.

Tim Kaddon was one of the people subjected to to our lego demands and wrote a great blog entry about his experience in the game.

Thanks to Conrad Bentham for organizing and facilitating the session, and Adrian Wible, Lefak Fakiyesi, and Brian Gunthrie for participating and making it fun.

Top Highlights:

Seeing how seriously people take building anything, including a lego animal.

Seeing the big Aha! moments, like where they realized they forgot to ask the customer what they wanted, or what the priorities were in the first iteration.

Me trying to find a balance between getting the lego animal I’m satisfied with, and torturing the team in order to make the game more interesting

Being reminded of how great a forum the game is for communicating a wide variety of Agile concepts.

I recently checked the statistics for my blog and I was surprised to find that one of top reason that someone came to my blog was because I had tagged a previous article with the “Thoughtworks interview process” or something to that effect. Unfortunately for those people, the article was brief, and didn’t reveal much more than the Thoughtworks interview info page.

Given that I have been involved in the interview process at thoughtworks, I thought I would share my perspective for anyone who’s interested – the perspective of a developer at Thoughtworks that may be interviewing you at some point.

General Considerations

Here are some general consideration that go through my mind when I’m evaluating a candidate.

1. Are you a top notch developer
2. How much do you know about Agile, and do you have experience in Agile
3. Are you a fit for consulting – ie: can you travel, will you be mesh well in a variety of client environments
4. Are you generally interested in Technology
5. Will we want to work with you, are you fun to work with and would I learn from you
6. Are you poly-skilled – will you be able to work in many programming languages, platforms, and frameworks
7. Are you open to learning / are you open to new ideas

Specific Tips

1. Spend some time researching us and have an idea what we care about. This doesn’t mean you have to know a detailed history of the company, but you should have an idea of some higher level points of who/why/what we are.

2. Take the code submission seriously. This is the one of the first major gates that needs to be passed to get to the interview process. It’s your chance to express to us your coding philosophy. Don’t communcate “I don’t care about this part” or “I didn’t have time to do it well.”

3. Don’t try to tell us what you think we want to hear. Don’t try to show that you have more knowledge/experience than you do; it’s very easy to spot that. Be yourself, honestly express what you know and don’t, and what you are interested in, or not.

4. Have fun. We hope to have fun on days were we interview people, and it’s great when the candidates have a good time as well.

5. Get a good nights sleep before the day of the interview. It’s going to be a long day, and you don’t want to try to swing it on 4 hours of sleep.

If you don’t get in, don’t worry

1. There are many points of failure – Our process is not easy. Unlike the way many other companies interview developers, candidates don’t just show up and talk to a few tech-lead or architect people. This means that if you have an “off day” there are more stages that it can affect.

2. Our process has variation – There is no perfect pass/fail criteria in the process. It depends on a myriad of factors that might be different on a different day.

3. Perhaps you just weren’t a good fit overall And that’s okay. No one is able to work anywhere. You are probably exactly what someone is looking for.

4. You might not be ready yet, but that doesn’t mean that you won’t be next time around. Sometimes, there are people that have the right background, and the right attitude, but they are a little short on experience or skill-sets.

I had blogged previously about how to win the game where I discuss the concept of game theory as it applies to software development. I outlined what I considered the highest level considerations, but I wanted to go into some more depth into how you become good at a game.

There is a particular quality to games to determine if you are winning them or not; When the game allows rapid cycles and fast feedback, you can figure out whether or not you can get better at the game. In the case that you aren’t getting better, you probably should not be playing the game, or you should change the game so the player can get better at it.

The last point is important, and one of more difficult considerations. Perhaps that team will never win the game. The only way to discover that is to play the game in cycles. These cycles provide the information needed. In the previous blog post, I outlined that there were 3 important aspects of a game.

1. Knowing how to win the game
2. Being motivated to win the game
3. Having the ability to win the game

This is all fine in theory, but how do you really know whether a team has unified these three considerations?

Back to my analogy to video games. I was playing a racing game, and at first, it seemed like there was little I could do to beat the computer players. The racing tracks were too hard, the computer players were too good. There was too much chance involved. Then, as I continued to play, I became a little better. Energized by the awareness of this, I played more, and got even better. After a while, I beat the game.

How did I know that I would get better? It certainly seemed like I was failing at first. The answer is that there were rapid cycles of practice and feedback. I got information about how I was doing.

How does beating a video game apply to the 3 principles that listed above?

The first is easy; I had to complete levels of the game that incrementally led to my winning the game. I clearly knew what it took to win the game. Each level that I won was not only a discrete step towards winning, but it was a test of my ability to progress to the end goal. I see a parallel here to having stories represent cohesive unit of valuable functionality or features.

The second is also somewhat easy; I wanted to win. I enjoyed playing the game. It was stimulating and rewarding enough for me to continue trying to win. However, I think it’s important to note that a large part of the reason that I maintained motivation was that I knew I was getting better, and I knew that the end goal was achievable. If I can’t win a game, I lose interest. If I were forced to play a game that I felt was losing, I would also lose motivation, even if you were paying me to play the game. This is reminiscent of a project death march.

The third point is perhaps the most relevant to this blog post. There would be no way to know if I was capable of winning the game without short, achievable, and valuable cycles that informed me of my progress. Not all games can be won. If you were to have me play a computerized chess game meant to compete with a chess grand master, I may never win, even if I play for years.

A difficult part of a project is determining if the people of the project have the ability to win the game. There are many reasons why the game is setup such that it can’t be won. It’s not just whether the people involved aren’t capable (wrong skill sets, insufficient experience, etc), it’s often because the structure of the project impedes success and/or the scope is too large.

I have a lot more to say on this topic. I think that the game theory analogy is very applicable to software development, and organizational behavior in general. I think that there is a fundamental project management question; How does one form a group of people to achieve any endeavor? The general answer to this question is applicable to software development as well.

Often a quick search on usages of a method or class is sufficient. You look at the tests briefly to see what is going on, and then you make a new test to assert the new behavior that is going to be added.

Sometimes, when there are a lot of tests, I just prefer to remove the code I’m going to modify, and see what tests fail.

This technique provides some quick and useful information. For instance:

1. What tests were written that test the code? Are there any applicable tests?
2. Are there other tests that implicitly test the code and should not?
3. Are the current tests testing all the conditions?
4. Is there a test that no longer applies?
5. Is there a test who’s expectation are changing? That test can be modified instead of writing a new test.

This approach tends to be practical in the case of unit tests, and perhaps a few integration tests that you may know are related. Otherwise, it becomes too expensive to run a long running suite.

An additional benefit is that if you already know of some tests that should be testing some code, you may find a test that meant to test the logic, but for some reason was always passing.

This made me think about a general organizational “smell” in software development. For lack of a better label, I’ll call it fragmentation of goals. A manager might be juggling multiple projects, of which yours is not the highest priority. QA’s may be measured based on how many bugs they find, or how many test cases the document. A developer might see a better chance at promotion by going to all the meetings he or she is invited to, moreso than the delivery of features. Entire teams within a project may even be motivated by goals that are not complimentary to the success of the project.

When I was in college, I liked to compare the team projects I was on to games. My goal was making sure that we were winning the game (in that situation, getting an “A”). Later in my education, I read Agile Software Development by Alistair Cockburn, and he made the analogy that software development is like a cooperative game that we are trying to win. I’ve always loved that analogy, and I think that it’s a useful tool in conceptualizing an effective team or project. Much like a team game, software projects must be a coordinated and unified effort amongst individuals to achieve a common goal.

There are three important factors that are core to a team being able to deliver software successfully, and each easily becomes fragmented (I’m generally using team and project synonimously).

1. Knowing how to win the game
2. Being motivated to win the game
3. Having the ability to win the game

I’ll discuss each inline below.

Knowing how to win the game

A team or project must know how to “win the game” – there has to be acceptance criteria on the results of the teams efforts. Not having this clearly established makes it impossible for people to unite towards winning (I am not implying having all the “requirements” – I am talking about a clear understanding of how to accomplish the business goal). Without a clear goal, a team can only work towards something that they hope is right, and everyone may not agree on what that is.

An important aspect of this strategy is to have a single dedicated product owner – someone who can provide a vision of the end goal, and be the final decision maker for what aspects of the product will achieve that goal. This will help not only knowing what to build, but have the ability to prioritize when to build something.

Being motivated to win the game

The second part is ensuring that the team is motivated to win the game. The team as a whole will suffer if the individuals are engaging in activities meant to promote their own goals over the goals of the team. There are many, many parts to this. My basic opinion is that measuring an individuals success should be based on their contribution to the team, and the teams success.

It is difficult to make sure that everyone has the incentives to be motivated. It is not enough to hire people and then put them to work on something. You must make sure that they are happy, and personally interested in succeeding. It’s especially important to make sure that the personal goals and desires of the individuals aligned with the success of the team.

This doesn’t just apply to individuals. Often large projects involves dividing the project into smaller teams. It’s not beneficial to set up teams with goals that allows them to succeed while the project fails. The concept that this works is often an attempt to suboptimize a system – the idea that if the individual components are succeeding then the whole will succeed. This approach is flawed and often leads to a team achieving it’s goals, even at the expense of the project.

Having the ability to win the game

The team must have the ability to be effective and able to win the game. Further, the team must be empowered to make changes to increase their ability to win the game. The easy example of a team not being able to win is that when the team is formed with people who’s skill-sets are not suited to success. For example, building a team of people that have never done web development, and asking them to build a sophisticated web application. Another easy example is creating unachievable goals for the team, such as having the scope be too great for the team to ever achieve.

More complicated scenarios involve putting a restriction on a team that they must use a certain language or technology, or to disallow some tools and technologies, without consideration as to whether it actually helps the team succeed. It could even be as simple as the developers not being able to get licenses for software that they need, or computers to support the development efforts.

This point is part of the reason that it is critical for teams to be responsive, continually improving, and honest about what will make them more effective and better able to succeed. It could be that 1 month into a project, the team already knows that it’s failing and it’s likely that nothing can be done to help. It could be that the team is producing features, but is losing a lot of time trying to communicate to disparate groups that they rely upon.

Designing the game

A final point here is that all of these factors are part of setting up the game in the first place. Often the decisions that are made early on in a projects life sets it up to succeed or to fail. In my experience, the structure of a software project tends to be an extension of the organizational behavior of the company. That structure may not be an ideal structure for a software project.

Regardless of how the project begins, it’s important to change how the game is played if it is necessary to win the game.

It was a fun application to write – It had a Java Swing front end with JNI to use portions of the win32 API and X-Windows API, and we wrote a video and audio server to distribute the view of the application and allow the users to talk to each other in on online group phone call while they were using the application.

There was one desired feature that we could not implement. We wanted to allow users to simultaneously enter text, and simultaneously control their own mouse cursor to interact with the application. We were completely blocked on this feature because it goes against a major assumption of all major operating systems – that one user will use the system at a time. For instance, it’s assumed that one window will have focus at a time. It is also assumed that there is only one mouse cursor at a time. There are even more subtle issues that come up, such as there being only one copy and paste buffer.

At the time, I was disappointed, but I didn’t dwell on it too much. I figured that that level of multi-user collaboration would only service a very small niche market, and that it wasn’t so bad that we didn’t have the possibility of better collaboration.

Fast forward to today. I pair program most of the time, and I find myself longing for an environment that would allow me and my pair to collaborate in different ways. It’s standard pair programming, where we both work on the same window, and have to share the same keyboard and mouse input (preferably with 2 mice and 2 keyboards). So what do you do when one person has to check email, or one person decides to research an issue online, or otherwise engage in a task that is not the same as the other pair member? Typically, each person will have a laptop that they can use for these situations, but that causes the person to separate themselves from the primary work environment. Sharing information will have to done by sending an email or putting a file on a shared drive, or having the other person look at information on the laptop’s screen. What about spiking a problem side by side, where each pair tries two similar but different approaches to solving a programming problem? I would prefer to collaborate with someone in the same environment, where we can do some different tasks in parallel, and can share windows, applications, and data in the same desktop environment.

I think that it would be interesting to expand this idea even further. What if an entire team worked in the same environment, on the same desktop? Each person could have a portion of a huge desktop, and people in a team could choose to collaborate with each other in a very fluid fashion by putting their desktop spaces close together. It would be an interesting experiment in collaboration applied to software development.

Even if multiple simultaneous mouse and keyboard input were generally possible in operating systems, there would still likely be a lot of complex concurrency issues for developers, such as making sure that servers bind to different ports and write out to different files. It’s likely that once popular operating systems generally allow the multi-user features I mentioned before, that it would still be preferable to have isolated machines that interact through collaborative software rather then have everyone actually use the same machine.

I’ve seen software that allows sharing of a the mouse and keyboard across machines, and sharing the copy and paste buffer, but the purpose was to allows one person to connect 2 machines and use them like an extended desktop. Perhaps software like this could be tailored for better collaboration for pair programming or developers on a team not pairing.

Basically, what I want is an inspection tool that also has a bundle in textmate, so that I can see the history of objects that are created, as well as interactions in a form that is searchable and that quickly shows what really happened. It would be like having debugging on all the time.

The goal of this tool would be to allow developers to view what really happens with the code when it executes. Sometimes finding the source of bugs is difficult because statically analyzing code is not the easiest way to predict the future state of an object, or to predict the interactions between objects. You don’t really know what happens until the code executes.

Nor is traditional debugging an efficient way to get the information that you need. You have to set a break-point, add in a line to call the a debugger, or even print out some information in order to answer a question about the actual execution of the code.

What I want is a tool that records a wealth of data about the actual execution of some code, such as when I run a test, or initiate an action from a controller. Then, I would like to be able to filter/query that information to get more precise information.

For the inspection tool, I’d like to know things like:

- See the actual methods and fields of a class or object instance at the beginning and end of code execution
- See all calls made to an object, or a specific instance of an object
- See all callers of a specific method of a specific object type or object instance
- Detect changes in the structure of an individual object that happens, such as methods being added or redefined after the object is initialized
- View all changes to fields of an object
- I would like to see the order that objects are created in. I would like to see when all objects of a certain type were created, or objects that respond to a specific method
- I would like to see all interactions between 2 objects
- I would like to be able to further scope the search, so that I only see information within the scope of a specific method call

Preferably, I would want to be able to view all of this information after running the code one time. And then be able to enter some search criteria into a form and have it reduce the amount of information to match my search criteria. Then, I can tweak some starting data in a test, and then rerun the test and get a new set of information to query.

I think that most of this information could be obtained by creating a Module to intercept all method sends to every object, and save some data about the context of the call when it’s made. Then, a client tool could parse the output, and allow a developer to query it.

I tried to do some googling to find something like this, but all I found were standard debugging tools, which I don’t think have the features that I’m looking for. If anyone knows of something that can do what I want, let me know.