Although there’s still a long way to go, we have learnt a lot, and we are getting a much clearer idea of where we need to go in future. And I feel that I am developing another startup skill which will no doubt be valuable in future.

I will be writing about our customer discovery process and things we have learnt in a series of blog posts on the Go Test It blog. A slightly provocative teaser from my first post of the series:

A startup’s purpose is not to make money (at least not initially). Nor is its purpose to build a product (although it won’t go far without a product). No, the real reason why a startup exists is to learn about potential customers and to define a market.

Start reading the new series on the Go Test It blog, subscribe to the Go Test It RSS feed and follow me on Twitter to keep track of new posts.

I’m sure much has been written about this already, but I’ve not really read any of it. Instead I made up some principles based on my own bit of experience and a bit of common sense, and maybe you will find them useful.

So let’s take a look at a negotiation from the point of view of the seller. When you’re selling something, you want to charge as much as you can get away with. The higher the price, the better for you, i.e. the higher the monetary benefit of the sale to you.

The first bit of money makes a big difference — maybe you really need it so that you can pay the rent, otherwise you’ll get kicked out of your house. Within this range, having more money is clearly a lot more valuable for you. However, as the amounts increase and you have secured an acceptable standard of living, I would argue that the benefit of the money to you slows down. Frankly, a top-paid investment banker would barely notice the difference if their salary or bonus was increased or decreased by 50%.

But nevertheless, although it gradually flattens a bit, the line is always increasing. If you can get more out of a deal, it’s a better deal. At first glance anyway.

However, that’s not the end of the story. When I am negotiating with someone, this is usually someone I actually want to work with, and I want to stay friends with them. You don’t want to charge so much that they will simply walk away. And if you’re in a strong position where the buyer doesn’t have much choice (eg. because it’s something urgent and they don’t have time to find someone else), you don’t want to abuse your position, as otherwise you will get a bad reputation as someone who takes advantage of others.

Therefore, you should be taking your relationship with the buyer into account. How indignant they will be about your price will depend on a lot of factors (the market value of what you have to offer, how much you charged them previously, how much they can afford, etc). But in general there will be some sort of function describing how happy they are depending on the price (BogoGraph alert!):

Here I plotted in blue the buyer’s “goodwill” (not in any financial sense, just in the sense of “how much they like you”). If you charge too much, they will obviously hate you, so the graph goes downhill very rapidly beyond a certain point. You probably don’t want to go there. I think it is also possible to charge too little, which conveys the message that what you have is not particularly good and not worth very much; but that won’t offend your buyer nearly as much as overcharging them. So I’d say that the graph first goes up a little bit, and then goes down a lot.

How do you figure out the shape of that curve for your buyer? That’s hard. You need to get a few data points, for example you might try asking them for a pretty high price and observing how upset they get. But you’ve got to be very careful — don’t abuse their trust and don’t waste their time. The longer you spend trying to measure the blue curve, the lower it gets overall (i.e. the more irritated they get with you). It’s better if you can use external things as reference points — how much an alternative solution would cost them, how much they have paid for comparable things in the past, the size of their budget (which they probably won’t tell you but you might be able to guess indirectly).

The value you get out of making the sale is twofold: on the one side the money you get out of it, and on the other side the buyer’s goodwill (with all of its intangible benefits, such as keeping open potential future deals, recommendations, referrals, general happiness and warm fuzzy feelings). You want both, so let’s just add the two curves together:

The red line is simply the sum of the blue and the black, and represents the total value you’re getting out of the sale. Some things to observe:

1. There is a very clear maximum for the total value. This is what you should be aiming at.
2. The price at which the seller’s value is maximised is always higher than the ideal price for the buyer (i.e. the red line’s maximum is further to the right than the blue line’s maximum). Of course the buyer would prefer to pay less — if that is not the case, you’re definitely charging too little.
3. The blue graph is like quantum mechanics: it changes when you measure it. Keep your negotiations short, simple and clear. Don’t keep changing your mind, because that just reduces goodwill and thus the total value. Keep in mind that the whole point of this exercise is to stay friends!
4. The optimal price depends on your weighting of blue curve vs. black curve — the less you care about the buyer’s goodwill, the higher the optimal price.
5. Even if you are playing hard-nosed, it’s a game of diminishing returns. By pushing harder you will get a bit more money, but the buyer will also get a lot more indignant. The harder you push, the smaller your marginal gains.

This brings us to an interesting concept: you can trade in buyer’s goodwill for more money by adjusting how much you care about how the other party feels. There is a return on the indignation of the other party, and it’s up to you to choose what you want this return to be.

It depends on your character and on the strategic value of the particular deal. I generally work with quite a low return on indignation, i.e. I value goodwill quite highly and won’t readily trade it for a bit more money. That’s because:

1. I don’t find it fun to be hard-nosed, so I’ll much rather be nice (assuming the other party is also nice), and
2. I think in terms of long-term value, and I believe that goodwill stays around for a long time, so I’ll much rather invest in building good long-term relationships than try to extract the maximum monetary value in the short term.

That’s my approach, and I’m sure others will think differently. But at least, with a framework like this, you can be conscious about your return on indignation.

By the way, if you’re into pricing products which are not individually negotiated — quite a different topic — you should definitely read Don’t just roll the dice by Neil Davidson.

Over Christmas I finally had time to read The Four Steps to the Epiphany by Steve Blank. I had heard from a few people that it was the best book in the world for startups, but of course you take that sort of recommendation with a grain of salt. When I finally got round to ordering it, my first impression was not very impressed. The graphics are misaligned, the typography is ugly, there are plenty of typos, the cover picture is cheesy, the CafePress binding is flimsy. All in all, not a good start.

Well, don’t judge this book by its cover. Despite those apparent flaws, it is absolutely brilliant. And yes, if you have any sort of startup ambitions, you should go out and read it immediately.

In fact, maybe the book is deliberately ‘unprofessional’, because that would be consistent with a theme which runs through the entire book: focus relentlessly on what really matters and what really adds value. What really matters to me with Steve Blank’s book is purely its content (which is clearly articulated and deeply insightful); professional design or editing wouldn’t have changed this book’s value to me. Similarly, what really matters with a startup is to discover and learn what customers need, how the product fits into their lives, and how you are going to get it into their hands. ‘Professionally’ executing a strategy comes later. First you’ve got to learn and discover what the strategy is going to be.

This sounds trivially obvious, but it is not.

Let me digress for a minute. Something else I read recently is The Fable of the User-Centred Designer by David Travis (a short but beautifully written eBook — well worth reading but quite different from the Four Steps to the Epiphany). It made me realise how badly we had gone wrong with Go Test It. Steve Blank’s book further strengthened that feeling. Ok, we built a product which works alright. We did a few informal usability tests (looking over people’s shoulder while they use it for the first time) and we got some useful feedback from the beta tests. And clearly the result was promising enough that Red Gate wanted to acquire it.

Here is my confession: I cannot truthfully say that we really engaged customers in the process. I had some ideas about use cases and I did a few pencil sketches of the user interface before it was implemented. But did I actually go out to potential customers and test my ideas on them? Not a single bit! We thought about the ideas for a few minutes by ourselves, nodded our heads, and then just went ahead and hacked it together.

I have no excuse whatsoever for ignoring our customers like I did. Hell, we even had a poster from the Usability Professionals’ Association hanging in our office for a while, detailing the steps of a user-centred design process. (Some years ago I thought our company was going to be a usability consultancy — that was before we got into web development and ultimately into building Go Test It. Hahaha! By the way, that’s why this blog is called Yes/No/Cancel.) And nevertheless I totally ignored it. We were not doing anything like user-centred design, let alone Customer Development as proposed by Steve Blank, which is a lot further-reaching.

The only thing which saved us was that I was basically building a product to solve my own problem. I had worked on a big, JavaScript-intensive web app project, and had felt the pain of getting it to work in different versions of IE. So I had an idea of the kind of tool I had wanted to make that project less painful.

So: building something which scratches your own itch is better than building something which you don’t even need yourself. But it’s still a pretty bad starting point, because you are only one data point. How do you know that you’re not an outlier? In our case, I was even a pretty bad data point. I had only worked on two significant commercial web app projects — not exactly a great deal of experience. I had never worked in a proper web agency, or a larger software company, or an established e-commerce retailer, or in fact any company which looked remotely like the type of company we’re trying to sign up as customers.

What we should have done — and I understand this now — is to follow a Customer Development route from the start, alongside building our product. Before the coding started, I should have at least made my hypotheses explicit, tested them on my target market, and refined the product idea. Basically, I should have read The Four Steps to the Epiphany a year ago and then followed it.

In my defence, it’s difficult when you are a sole founder. In principle you could multitask between Customer Development and Product Development, but I think the two activities require very different mindsets, and the context switching overhead between the activities is huge. Therefore I suspect that a sole founder doing both will take much more than twice as long compared to two cofounders who specialise in Customer Development and Product Development respectively. Hence it’s extremely tempting for a technical founder like me to pretend that the Customer Development side doesn’t exist, and focus exclusively on the product.

Well, late insight is better than never. Amir is joining me on the Customer Development side of Go Test It, and we have a lot of catching-up to do. In 2010 there will be soul-searching and maybe some changes of plans, but I am really looking forward to it, because I am confident that we can figure out how to turn Go Test It from something ok into a product which you simply must have.

Today we announced that we are selling our startup to Red Gate. Investors refer to this as an exit: at this point they typically sell their shares, so they get the money back which they invested some time before — or rather, they are supposed to get quite a bit more money back. I had one investor, Peter Cowley, and in total, he got more than 15 times his money back within 2.5 years: that’s a hell of a lot better than if he had stuck it in a savings account!

A few exits are huge (think Google buying YouTube), many are medium-sized, and many are small. This is one of the small ones: although I’m getting some money out of it, it’s certainly not making me rich. However, I am very happy with it: the hard work of the last few years is paying off.

For the founder (that’s me), it’s not really an exit: I have to and want to continue working hard on Go Test It for another year or so. As Simon from Red Gate put it, they need me to continue making the product better, and they need time to get everything out of my head and into someone else’s head. (Do they really want everything out of my head? Even those childhood memories of listening to the Queen cassette? Well, if they want them, they can have them.)

Anyway, the reason I started this post with a bout of nostalgia is that this is pretty much how I feel today. It’s like a 10-year-old proudly playing back his first attempt at making spooky ghost noises on a cassette recorder. He knows that it’s not perfect, and that his work will change the world only a tiny little bit, but he learnt a lot, and right now, it feels like the greatest success and the most important thing in the world.

The aspect of learning is probably the most important one for me. I learnt an incredible amount by starting a company and taking it to exit, and I expect to learn a lot more over the coming year: how to take a product from “it works pretty well” to “it is absolutely amazing”; how to work with more than the bare minimum resources I have been used to; how to scale the business, attracting and retaining lots of happy customers.

That name “My first Sony” is evocative: it is oriented towards the future. The “first” implies that there will be a “second” and a “third”; of course they want you to buy many more Sonies during the rest of your life. But not only that: the “first” of something signifies a turn of events. From now on you are sonified. Before its first Sony, the child could only store sounds in its head; from now on, it can record them on cassette. Before, it could only listen to music and audiobooks when the parents gave permission to use the hi-fi; from now on, all available cassettes can be listened to at any time. Even secretly under the blankets.

My first exit is a bit like that: it also is oriented towards the future. I can’t predict what the future will bring, but evidence suggests that entrepreneurs tend not to be satisfied with just one successful startup, but want to try it again and again. But even if I do start another company, and even if I do have another exit, there will always be something special about that first one. The one which proved that I can do it.

Of course, saying that I built this company is about as true as saying that Vespasian built the Colosseum. I bet he didn’t lift a single stone himself. Therefore, at this point – clichéd though it may seem, I mean this very genuinely and sincerely – all I want to say is a huge round of thanks to everyone who has helped me along this journey. None of this would have been possible without you. Thank you to:

• Simon and Neil, joint CEOs of Red Gate: you have built a company with a fantastic culture and great people, and I respect you deeply.
• Peter, my mentor, advisor, investor and friend;
• Conrad, Sam, James and David, who helped build it;
• Rahul, who helped evangelise it;
• Patrick, who came up with the name;
• all the open source projects which are the foundation of our work, particularly Selenium, Windmill, Ruby, Rails and Scala – we will show our gratitude by contributing as much as possible back to the community;
• the Accidentals and the Springboarders, who form such a supportive startup community;
• our clients, whose business funded much of the development of Go Test It to this point;
• all the Go Test It trial and beta users, for your valuable feedback;
• everyone who has supported, encouraged and inspired us;
• my friends, for bearing with my obsessive-compulsive attitude to work;
• my family, because no mountaineer can reach the top without a solid base camp;
• Rita, for your endless supply of smiles.

…

Ok, well, that was nice and romantic. Anyway – let’s get back to changing the world!

The interview is for the Startup Success Podcast, a series of shows providing a wealth of useful information and inspiration for startups. In this episode, Patrick Foley talks about his visit to the Microsoft Professional Developers Conference (PDC), and I talk about Go Test It – what it is, how it works, why we built it, where it is going in future. There’s even a special discount in there! :)

Head over now to the Startup Success Podcast and listen to the episode! (The interview with me starts at about 15 minutes in.)

Ok, the elevator pitch is pretty straightforward: Go Test It helps web developers to test automatically whether their site works correctly in different browsers. The technology works well and lots of people are using it successfully.

But how much does that really tell us? Is it really the web developers who need us, or should we aim at the testers, the user experience managers, the web content administrators, or the sysadmins? Are those people in big enterprises, small web consultancies or are they freelancers? Do they use PHP, Ruby on Rails or ASP.NET, and which JavaScript framework to they use? Do they have a dedicated QA team or do the developers do the testing themselves? Does their work have to be signed off by a client or are they doing it for themselves? How do they currently test their application, and how would we fit into their workflow? Do they develop websites for a particular industry, e.g. education, finance or healthcare? Do they need a big consultancy contract, do they require the product to be customised, or do they just want a straightforward self-service sign-up over the web?

It is simply not possible to serve all different market segments and niches at the same time. You’ve got to choose: resources are limited, and you’ve got to pick your battles wisely. But I am guilty of not properly thinking through the segmentation and figuring out the best way of targeting a particular niche.

For the Business of Software 2009, Joel Spolsky organised three startup workshops aimed at covering the main areas of difficulty faced by software and web startups: marketing, product pricing, and finding your first 100 customers. Exactly those areas in which I am guilty of negligence.

Neil Davidson and Simon Galbraith of Red Gate were in charge of the “finding your first customers” workshop, and they asked me to help them design the workshop. Since my startup is one of the companies in Red Gate’s “accidental incubator”, and it’s at the stage of finding the first 100 paying customers right now, Neil and Simon suggested that we use Go Test It as a case study for the workshop.

What we wanted to do is to ask a number of key questions about the market positioning of a product and aspects of the sales process — crucial questions for startup founders, but also questions which are easy to ignore if you focus too much on the technology. By discussing a case study in the workshop, the participants could practise thinking about and answering these questions in a fresh and unfamiliar context, and develop tools and techniques which they can apply in their own companies. We chose questions that are relevant to any early-stage B2B software company.

The workshops worked out really well, and so we would like to release the materials we created for this workshop under a Creative Commons license and make them available to startups everywhere. You can download the worksheet (PDF). I have also written up how we structured the workshop and what we learnt in the process.

Below I have added some notes to clarify what we mean with each of the questions. I give several illustrative answers for each, but please remember that they are not multiple-choice questions. The whole point of the exercise is that you come up with your own answers!

1. Sales objective: what are you hoping to get out of your customers?

   At first sight this seems a pointless question: of course you want your customers’ money! However, think about it a bit harder. Maybe getting some great case studies and testimonials is worth even more than the money, because they will help attract more of the right sort of customers. Maybe you really want feedback to help improve the product (feedback from paying customers is worth much more than feedback from free-riding users, because paying customers really care!), or maybe you want to learn about the sales process — what does your customer’s org chart look like, what are their key external relationships, and who are the key people you need to get onto your side? The things you can learn from a customer are potentially worth a lot more than the cash they hand over to you.

2. What is the pain point addressed by the product?

   A standard question, but worth revisiting from time to time, because the answer may not be as obvious as you think. In the case of Go Test It, the obvious answer is “needing to make sure that the site works in all browsers”. But ask yourself: why does it need to work in all browsers? To provide a good experience to all users, so that they continue using your site even if they are using an obscure or buggy browser? Or maybe it’s a matter of meeting a compatibility warranty given to a client, or passing certain acceptance tests? Is it a tool for people to cover their back within their organisation?

   Or maybe the customers currently do manual cross-browser testing, and their real pain point is actually the release cycle duration (very slow in manual testing), or the staff requirement (manual testing is too expensive), or staff motivation and turnover (manual testing is repetitive and boring, leading to low job satisfaction)?

3. What is the sales process?

   Startups often worry a lot about marketing (figuring out who to target and how to get on their radar), but neglect the following step: now that you’ve got people’s interest, how do you convert them into paying customers? Giving them a form to enter their credit card number is well and good, but what about people who trial the product but never quite get round to signing up to the paid version? Will some follow-up emails be sufficient, or do you need telesales people to get on the phone and talk to the customers? Do the customers require help importing their data into the system or even an integration project? Maybe you need to give demos or seminars at customers’ premises, or even learn to play golf?

4. What do you want your marketing to achieve, and how could you measure it?

   Your end goal is probably “get lots of paying customers”, but there are various possible intermediate goals along the way. Depending on your strategy, different things may be important. Do you want to maximise the number of free trials to which people sign up, or the number of newsletter signups, the number of Twitter followers or community members? Or would you rather focus on a small number of customers and optimise for high-quality leads and maximise level of usage? Do you want to go “viral” and maximise the number of referrals from each customer while minimising the time to referral?

   Intangible but maybe equally important aspects of marketing might be promoting a positive brand image, building a good reputation, and encouraging users to become passionate evangelists for the product. You might still try to measure these kinds of things by monitoring Twitter for keywords and informal surveys, but they are difficult to measure quantitatively. Whatever you want your marketing to achieve, remember that your resources are limited, and hence you should probably focus on two or three achievable and measurable goals.

5. What will you do to achieve your marketing goals?

   This is a question about marketing channels: how are you going to get noticed? Social media and online communities, comprising blogs, Twitter, LinkedIn, Facebook etc. are probably the most hyped these days, and for some market segments they are indeed an excellent marketing channel. But you could also consider networking in the right circles, PR (including connecting with influential bloggers and getting them to write about you), attending conferences and trade shows, creating publicity stunts, competitions and much more. Hey, even advertising isn’t dead yet.

6. How will you allocate money and time over the next 3 months?

   This does not have to be a detailed budget, but a quick sketch of a pie chart will help you to think consciously about your priorities and make them actionable. How much of your resources should be allocated to engineering, sales activities, various marketing channels, learning from customers, etc.?

7. How will you price the product?

   It’s easy for engineers like me to spend a lot of time thinking about pricing, because it’s more within our engineering comfort zone than most of the other questions on this worksheet. Yes, it’s important (that’s why we included it), but only in conjunction with the other answers. This is our only multiple-choice question, encouraging you to answer it quickly and move on.

   Do you want to publish the pricing on the website for everybody to see, or do you want prospective customers to contact you for a quote? Will you have a fixed price list, or will you negotiate pricing individually per customer? And the most tricky: if you can’t do flat-rate pricing, how will you split pricing into bands? By number of users? By length of tests? By level of usage? By availability of features?

8. What will you change about the product?

   Once you have settled on a particular market segment and strategy, certain limitations in the product will become apparent: maybe some additional features are needed, maybe the technology needs to be presented differently in order to fit with the users’ point of view, maybe some unnecessary features can be removed. The important point is that product changes should be informed by the market, not just a preconception of what users will want.

9. Top 3 things you must do in the next 3 months?

   This could be anything: marketing, sales or engineering activities. The key is just that these tasks must be actionable: it must be clear how they can be done and they must be feasible now. “Build a community of 1,000 people working in this field” is not actionable, but “set up a Facebook page, blog about it and invite all of our friends to it” is.

10. If your users are superheroes, what is written on their T-shirts?

    As a final question, remember that users are human beings too. They don’t particularly care about your product as such, but they probably care about things like doing a good job, looking good with their peers or boss, having a sense of belonging, getting to go home early, and making a difference. Your product should turn your users into superheroes: give them exceptional powers which make them proud! So proud that they will want to wear your product motto on their T-shirt. What will it say? (Hat tip to Kathy Sierra, inspiration for this question.)

If you can answer all of those, you will have covered a lot of ground and will have a pretty good idea of how your product fits into a particular market segment. And in case that wasn’t enough, here are three bonus questions from our drafts which didn’t make it into our final worksheet:

11. Which key factors determine whether a potential customer will convert?

    This question complements the sales process (question 3 above). If the sales process is about the actions you can take to convert interested people into paying customers, then this is about setting the right environment for conversions. What do customers care about? Is it the ease of integration with their workflow, the user experience, the amount of learning required, the level of support, references from other customers, key features, integration with systems they already use, reassurance that your company is not going to disappear overnight, or anything else?

12. What underlying need is satisfied by the product?

    We have already talked about the pain point above (question 2), which defines why there is a need for your product. However it can be insightful to look behind the pain point and figure out why it exists: the “need behind the need” in the words of Paul Kenny. When a customer buys this product, are they really buying reassurance (security, peace of mind, covering their back)? Or are they buying ego (competitive advantage, ability to overcome difficult problems)? Are they interested in convenience (making their life easier), saving money or reducing risk? This underlying need should determine the light in which you present your product and how you pitch it.

13. What would you consider to be success, and how would you decide whether to abandon the project?

    This is an interesting one: not many people talk about criteria for success and failure. Usually success is unbounded — you always feel like you could do better, no matter how well you are doing. And failure is something that is not talked about, as if merely contemplating it would bring about a jinx or induce an unwanted pessimistic vibe. However, if you have a quantitative criterion for deciding when to cut your losses, you avoid giving up too early just because you’re feeling down.

With all these questions, keep a few principles in mind:

• Resources are limited, so you should focus on the things which are likely to be most useful.
• “Just try it out” and “talk to our users” should be part of any strategy, but they are not strategies in themselves. You’ve got to know what you are looking for and what questions to ask the customer.
• Argue why the actions you chose are the most appropriate, given the product, the market and any constraints.

Download the worksheet (PDF) and an example we filled in (a contrived example based on the fast food industry — not model answers, just to illustrate the kind of answers we’re looking for). Please also see my separate post on how we ran the workshop.

Did you find this useful? Please leave a comment, subscribe to my RSS feed or follow me on Twitter!

Photo credit: John of Austin on Flickr; Creative Commons license.

Update (2009-12-18): Added scanned example worksheet.

The workshop was aimed at startup founders and was centred around the topic of finding your first 100 customers. There were also two other workshops: one by Dharmesh Shah on marketing, and one by Joel Spolsky on product pricing. Three rotating groups of startup founders participated, and each workshop happened three times, so everyone got to go to each one.

I was to help Simon and Neil, who wanted to strike a tricky balance in their workshop: they wanted maximum interactivity for the participants, but also provide enough structure for the workshop to make it more than just a chat. Together with Amir, and with advice from Pam of Innovia Technology we came up with this workshop format:

• In a workshop of 20–24 people, allocate people into 6 teams of 3–4 people each. That’s the ideal team size: large enough to get an interesting discussion going, small enough to give everybody plenty of “airtime”.
• The 90 minute workshop is split into about 10 minutes introduction, 45 minutes work in the small teams and 25 minutes presentation of results and discussion. (Yes, the numbers don’t add up. Some time is always lost in general faffing.)
• Each team analyses the same product, but applies a different strategy. That way, when results are presented at the end, everybody is on the same page, but interesting discussion points arise from different teams taking quite different approaches to the same problem. Neil and Simon suggested using my startup’s product Go Test It as case study since it is one of the companies in their “accidental incubator”.
• To structure the group discussion, we give each team a worksheet with ten key questions. The worksheet is printed on large-format paper (about A1 poster size) so that it can be filled in with marker pens and all team members can sit around it easily.
• Neil, Simon, Amir and I rotate around the room as facilitators. We don’t just ask “is everything clear?”, but stay long enough with each team to get involved in the discussion. If we feel they are getting side-tracked or missing an important point, we inject suggestions to get them back on track.
• There should be enough time for team members to get to know each other; the workshop serves a networking purpose as well as an educational purpose.
• Participants should learn from each other and develop new ideas and insights through bouncing thoughts off each other, not just off Simon and Neil. The Business of Software conference attracts many bright and motivated people, and with the right framework in place, there is a potential for the workshop to be a lot more than the sum of its parts.

We had put a lot of thought into the worksheet beforehand, trying to make it a comprehensive but concise summary of the key ingredients to successfully marketing and selling a product. We found that it worked very well, but we had to be very careful at the same time: those boxes on the sheet, designed with open-ended questions to gently lead the direction of the discussion, could easily become a counterproductive limitation.

It’s strange how people fall into roles depending on the context: when faced with a worksheet and boxes to fill in, I could see some people instinctively falling back into a primary-school, do-what-the-teacher-wants-me-to-do kind of thinking. I don’t blame them; in a similar situation I would have probably behaved like that too. Fortunately, when we explicitly emphasised that we wanted people to brainstorm freely, to (literally) think outside of the box and not to take the questions too seriously, that barrier fell away and the ideas started flowing.

Some teams even redefined some of the questions, which is great — startups should always be questioning any apparent rules! But the worksheet was nevertheless helpful, because it gave a structure and a goal to each discussion, allowing the teams to maintain a swift pace and avoiding getting bogged down in minutiae.

As an aside, we asked each team to come up with a name for their team, and also to put their real names on the sheet. It’s a simple psychological trick which encourages the participants to take ownership of their work and thus care about it more.

Defining different teams’ strategies

We asked each team to consider Go Test It as a case study, and to examine different strategies for marketing and selling it. In principle, the technology is appropriate to any web development project, but the world of web development is too big as that we could market to all of it at once.

How do you segment the market and find a niche to focus on? Well, we came up with six different dimensions which you could use for segmentation:

• Segment by technology (e.g. using .NET, PHP, Rails, Django or Java on the server; and using jQuery, ext.js, YUI, Dojo etc. for the JavaScript)
• Segment by job role (e.g. developers, testers, user experience designers, web content managers, product managers)
• Segment by company type (e.g. agencies and outsourcing companies, startups, web-based software companies, non-software companies)
• Segment by company size (e.g. individual freelancers, <5 people, <20 people, enterprises)
• Segment by level of service (e.g. hands-off self-service, per-customer customisation, big consultancy projects)
• Segment by industry (people developing web sites e.g. for the finance sector, for healthcare, or for education)

We allocated one particular segmentation to each of the six teams, and asked each team to pick one particular segment based on their experience and interest. For example, we would allocate “segment by company type” and they might pick e-commerce retailers as their particular segment. That choice then determines their strategy for the rest of the session.

This approach of letting teams choose their own segments worked very well: there was of course some overlap between different teams’ results, but we also saw a broad spectrum of different approaches, including some really creative ideas. This allowed individuals to play to their strengths and bring in experience of their own businesses’ market segments, while at the same time keeping the workshop interesting and inclusive to everyone. I think we managed to strike a good balance here.

If I have a chance to run the workshop again, I think I will focus on improving the discussion phase of the workshop: potentially we could have made more of it by contrasting different approaches, and figuring out why some actions are appropriate to some strategies but not to others. Of course, you never know what the teams are going to come up with, so this is hard to plan in advance.

All in all, I think it was a very useful workshop for everyone, and the feedback we received was uniformly positive. If you have any thoughts, please leave a comment or drop me a note on Twitter.

Go Test It now has its own Twitter account and a shiny new blog, to which you can subscribe for product updates and useful testing tips. I will keep this blog as my personal space, for general thoughts on software development, and I won’t inundate you with marketing messages, honestly! :-)

On a personal level, you might recall that I was going to start a PhD around this time. That was back at the beginning of this year, when I didn’t know whether Go Test It was really going to go anywhere.

Well, it turned out that Go Test It is now going really well, and doing it part-time alongside a PhD is just not going to work. We need to keep our competitive head-start and continue innovating; we need to support our customer base and work on growing further; we need to continue working on the product to make sure it is absolutely rock-solid and delightfully simple to use. It is important that we continue going ahead with full force.

I have therefore decided to withdraw from the PhD, and to work on Go Test It full-time. Academia can wait; customers can’t. It is great that I can now focus fully on making Go Test It the obvious choice for web developers’ testing needs.

In his 2004 short essay The Python Paradox, PG argues (perhaps controversially) that a company can hire smarter programmers if it chooses to write its code in a “comparatively esoteric” programming language. At the time, Python was probably considered by most people to be esoteric in comparison to Java — in the sense that not many people would learn it at university or for career purposes. Therefore, the programmers who knew Python were people who learnt it for fun; and learning languages for fun is an activity which typically only the bright and motivated people engage in. Which makes the language a good “quality filter” for people.

Of course times have moved on, and Python (and Ruby, for that matter) are definitely entering the mainstream. They are still fine languages, but they no longer carry as much of an early adopter aura about them. The culture of testing and code beauty which is embraced by the Ruby and Python communities is still something special, and that is now the primary reason why I would choose Ruby and Python over Java or C#.

PG’s observation still holds true. But what are the new technologies to look out for? What can you find in 2009 which has the same role as Python did in 2004?

Choosing a programming language

A few months ago, when I was designing the system architecture for Go Test It, our awesome cross-browser testing product, I had a lot of design choices to make. For some parts of the system it was fairly clear what we were going to use: for instance, the frontend web application was almost certainly going to be Rails, because I had some existing tools like the invoicing gem which I wanted to reuse, and because it was simply a good fit for the job.

But what about the actual test management infrastructure?

I chose to write it in Scala. There were plenty of reasons why this could have seemed a bad idea:

• I had hardly any experience in using Scala, so I would have to invest a lot of time learning it as I went along;
• nobody else I knew had any experience in using Scala at all, so anyone I hired would probably also have to learn it (at the cost of slower progress);
• some of the tools (particularly the Eclipse plugin) were still packed with bugs;
• although I had access to Java libraries, I knew that I would have to write library wrappers of my own to make use of the Scala language features;
• setting up a proper build process was pretty horrible.

Scala also had some technical merits; particularly the actor model for multithreaded programming was a good fit for our problem. Still, in a purely technical consideration, it probably wouldn’t have made much sense. But my heart still said yes. The prospect of working with a language which looked pretty, had static type inference, nice functional programming features, and runs on the JVM (battle-tested and optimised over many years)… my heart just said yes. It wasn’t a rational choice, but an instinctive, emotional one.

Paul Graham was right

Shortly after working this out and drawing my architecture diagram (a pretty insane-looking tangle of boxes and arrows on a sheet of paper), I was talking to Sam Stokes, a freelance developer sitting next to me in our shared office. Sam is bright, motivated and interested in the things going on in the software world. We were talking casually about Go Test It and I showed him my messy diagram.

And when I had talked him through the architecture (“I think I’ll write these three components in Scala, and they will talk to each other via a RabbitMQ message queue“), Sam said, in effect, “Hey, this is cool, I want to work on this”.

And he did. Over the course of a few weekends, Sam spent several days learning Scala and contributing to the Go Test It codebase. When you use Go Test It today, you are using the Scala code he wrote. He did this simply because he found it interesting and wanted to learn something new. The code he wrote was good, production-quality stuff. And he didn’t want a penny for it.

It’s not often that you get high-quality contributions to a closed-source project from a developer who is busy enough with several other projects already, for free.

Of course Sam cannot live on technical stimulation and my gratitude alone, and since then, we have actually contracted him to do paid work on Go Test It. But I found his initial reaction, and his approach to the project, a great example to prove Paul Graham’s point.

I would say Scala in 2009 has the place which Python had in 2004. There are a few other languages I would also consider for this title: Haskell, Erlang and Clojure come to mind (but don’t take that as a recommendation — I’ve not yet used any of them seriously myself). What these languages all have in common is that they’ve been around for long enough to come out of the purely academic space, are stable enough to be production-worthy, but are also new and exciting enough to attract good developers.

Fashion-Driven Development?

In an article about non-relational databases which Ryan Carson asked me to write a few months ago, I suggested that fashion can and should play a role in choosing which technologies to use. I got some criticism for this remark, but I still stand by my view. It is effectively a different way of looking at PG’s statement, provided you look for the fashion in the right circles (i.e. amongst experienced developers working at the cutting edge).

I wouldn’t go with fashion against all technical merit, but provided the technology is suitable and won’t increase your costs disproportionately, why not do something fashionable and adventurous? In an innovation-based technology business, the quality of your developers is key. Investments into things which make your good developers happy will pay off handsomely.

Demoing alongside me were four other startups: CamVine, mbed, CycleStreets and WebTicketing. I thought that all of them had very slick, impressive and credible demos (and judging by the positive feedback I got, my own one seemed to go down quite well too). I had not seen most of them before, but the message they got across in just seven minutes was so powerful that I’m immediately convinced of their potential, and since the demo night I’ve even been going around telling other people how awesome these startups are. That’s powerful stuff.

So how do you make a demo which will convince others that you are great and that they should tell all their friends about you?

Well, first you actually need something substantial to show. A product, a web application, something which is new and interesting and solidly built. I don’t believe in vapourware demos; you’ve got to show the real thing. However, for most companies I know in Cambridge that’s really not a problem — many have very impressive products, they just don’t know how to communicate them.

Even if you have a great product, you can still wreck it with a bad presentation. I’ve seen that happen and it feels tragic, so please do yourself and your audience a favour and make it a good demo. It’s not that hard, and these are the guidelines I set myself when preparing my demo. They may not work for everybody, but they worked well for me.

1. Know what you’re going to say

When I was at school, the only role I was allowed to play in the drama club was to be the lighting technician at the back of the room — no chance anyone would let me near acting. That’s because I can’t remember scripts word-by-word for the life of me, and even if I could, I feel really silly when reciting learnt lines. Therefore I have no choice but to speak freely.

However, you do need to have a pretty clear idea of what you are going to talk about, and how long it is going to take, if you want your presentation to be slick. You do need a script in the sense that you need to decide beforehand:

• exactly which buttons you are going to click, and
• which topics you are going to address.

Then speak the whole thing out loud twice. I find twice is a good number because I get a good feeling for how long the demo is going to take — essential when the presentation is strictly timed! — but I don’t feel over-rehearsed and therefore silly either.

I like things to start with a few slides introducing the issues your product solves. This gives the audience a context and makes your solution look stronger. But I wouldn’t spend more than 20% of the time in Powerpoint/Keynote mode; for example, in a seven-minute presentation, you should be in the actual demo after about 80 or 90 seconds.

Embrace these time constraints. They force you to be clear and to-the-point. Twitter is so successful precisely because it forces you to be short; it wouldn’t have worked without the 140 character limit. The same is true in presentations and demos.

2. Make it work offline

Don’t underestimate this issue. There are almost always technical problems in any venue, and most usually they are either bad internet access or problems with the projectors. Bad internet access is the most common. There’s no point in relying on a 3G card either, because probably reception will be bad in the Faraday cage that is your demo venue, and if not, the network will be overloaded because everyone is tweeting. There’s no way round it: your app has got to work offline.

And even though you probably have a full development environment on your laptop on which you can run your app, there are plenty of places where you’re accidentally relying on internet access. Some of my favourites:

• You’ve copied and pasted the Google Analytics/UserVoice/whatever JavaScript snippet into your page. When you’re offline, that script won’t load until the network times out, and so the browser’s page load event doesn’t get fired until 30 seconds later. But all your funky graphical effects only get initialised when the page is fully loaded, and suddenly you have no animations, no Ajax, nothing! It’s almost like the 1990s!
• You want to demonstrate clicking a link from an email, which opens a particular feature in your application; however, that email you sent to the test account contains the production site’s URL, not the address on your local machine.
• You are relying on hostnames provided by a DNS server on another network.
• Some easily forgotten part of your infrastructure (your mail server, memcached, message queue, mashed-up web service, …) is not on your own machine.
• Your Delicious Bookmarks Firefox extension (or other) tries to connect to its home server via an SSL connection. Since you’re a new guest to the network, the wireless access point intercepts the connection and inserts its own login page; of course its SSL certificate is invalid though. The result? An annoying SSL certificate warning dialog which pops out of nothing every two minutes.

To make sure your application works offline, fully disconnect the network on your laptop (or even better: go to a café with a dodgy public WiFi that intercepts SSL!), clear your browser cache, and make sure every step of your demo still works flawlessly. Make sure you leave enough preparation time to fix things that don’t work — it took me the best part of a day to make Go Test It work without an internet connection! And I had to use, believe it or not, various entries in my /etc/hosts file and even an ssh tunnel to localhost (for remapping ports). Yes, a bit crazy, but it worked.

In the demo night on Thursday we actually had the projector problem rather than the internet problem. But you’re on the web, right? It shouldn’t be hard for you to demonstrate your application on someone else’s computer. Even the hardware guys, mbed and CamVine, pulled off their demos smoothly using a computer they had never touched before!

3. Make it 100% reproducible

An interesting demo probably involves manipulating data somehow, and once data has been manipulated, it usually doesn’t like going back into the state it previously was.

You, however, should be able to run your demo several times, doing exactly the same steps each time. That means after each run of your demo, you need a way of quickly and reliably getting back to square one. And of course you need a well-defined square one in the first place.

Your development database will probably contain lots of edge-cases: excessively long strings, weird international characters, and other oddities from the times when you’ve been testing error handling. That’s not what your typical user’s data would look like, so I suggest having a separate database for demo purposes, with selected and nice representative example data.

Then you must write down the steps for getting back to the starting state you expect at the start of your demo: the database records you need to delete or add, the files you need to restore, the sessions from which you need to log out, the cookies which you need to clear. Or even write a shell script to do that for you. The last thing you’d want is that you can’t demonstrate a key feature of your application because some old data from your last trial run got stuck in there!

Another thing which you should write down, if you haven’t already, is exactly how to get your development environment up and running after a reboot. For a full Go Test It development setup I need about a dozen terminal windows open, each running a different process with an arcane command line, each providing a different part of our infrastructure puzzle. When I’m under stress I probably won’t remember every command correctly. They are all documented, of course — on our wiki, which is only accessible on the web. Print it off beforehand.

4. Clean laptop setup

If you’re anything like me, you’ll have tons of files on your desktop, a whole host of applications running, alerts popping up from time to time, and more. Some of this might be confidential. None of this should be appearing on the large projected image of your screen.

The easiest solution, I find, is to have a separate system user account to which I log in only for demos. It is stripped down to the minimum:

• blank desktop,
• menus only contain the apps which are absolutely required,
• no unnecessary Firefox extensions (see the Delicious problems above),
• screensaver disabled.

When using the second user account, you can still stay logged in to your primary account at the same time, and have all your terminal windows and stuff open there. It makes sense though to shut down all applications which are not required for the demo: TweetDeck, for example, is a notorious memory hog, and it would be a shame if one of the pages in your demo suddenly takes 10 seconds to load because a shortage of memory caused Firefox, your application server or your database to be paged out to disk.

Also keep in mind that projectors typically have a resolution of 1024×768 — make sure you’ve tested in advance what your website looks like at that width.

If you need to press arcane key combinations on your laptop to enable an external monitor, make sure you know exactly what those keys are, and that you’ve tried them beforehand.

5. And now, enjoy it

With all that preparation, there is nothing which could possibly go wrong. Sit back, take a deep breath and relax. Public speaking is an honour and a privilege, something to be enjoyed and relished, not something to be nervous about. Do your best to be focussed and relaxed, speak loudly and clearly, and smile at your audience. And it will work like a charm.