Brad Fitzpatrick, born in 1980, started to learn programming at the age of 5. In high school he went on to create a voting booth script called FreeVote, which he says earned him as much as 27 cent per click on banner ads back then (making for 25, 27 grand per month).

I touched my first computer (for playing chess) at the age of 14. I touched my first mouse (at a science fair) at the age of 19. I wrote my first C program at the age of 21. I have done a lot of programming since then, but the 16-year difference is a huge gap.

The reason is that my family and I didn’t have the money to buy a computer, nor did the schools and under-graduate colleges I studied have them for programming purposes. So I had to wait till my graduate studies. Those institutions have the right facilities today, but back in the 1980’s and 1990’s, India was just emerging from the lost years of the License Raj. The cohort I belong to share the same experience. We probably read a lot more books than a similar American demographic, but programming, not so much. (I remember reading a book on BASIC, but no computer to type it on.)

Time equals knowledge. A vast majority of the senior Indian programmers had their first experience with software development in college. While looking at reasons for the relative low proportion of software products coming out from India, this must be kept in mind. The Indian developers did not know enough programming to start a company or write great software during the early years of their youth, which are usually the most productive.

This is rapidly changing now. With the tremendous improvement in living standards over the past two decades, millions of Indian children are being raised in families that have “the” hardware! The gap in software innovation and entrepreneurship will surely decrease as these children grow up to become software engineers.

Step One, I think, is to pluck off our biggest competitors. We’re pretty certain that we’ve already built a great product that meets our customers’ needs — but there are still too many cases where we find out that, for some reason, someone went with the other guy. So that’s the development team’s mission for 2010: to eliminate any possible reason that customers might buy our competitors’ junk, just because there is some dinky little feature that they told themselves they absolutely couldn’t live without. I don’t think this is going to be very hard, frankly. Developing great software is something I’m pretty sure we’re good at.

Step Two is something I’m not particularly good at. Not in the least bit. We have to build up our sales force. The bottom line is, we just don’t do enough selling. I’ve been working on the assumption that a product naturally creates demand for itself and the sales team just helps fulfill that demand. But I’ve realized that I have things backward. I’ve come to understand that a sales team drives demand. My problem is that I’ve never been able to figure out how to hire good salespeople. For a guy who wrote a book on how to hire great programmers, it’s mortifying how incompetent I’ve been at enlarging the sales team, which, right now, consists of one terrific account executive and a dog. (I’m just kidding. There’s no dog.)

I will tackle the slow growth/slow death issue in a later post, but what does it mean when you want to try to rapidly accelerate the growth of your business, especially when it is sales-driven instead of marketing-driven, as in the case of FogCreek? Is it just about taking more risk, as Joel seems to think?

Risk-taking is a big part of higher growth, as you have to spend money on sales personnel, marketing, customer service and technical staff while you are uncertain about the returns from such investment. Sometimes, spending money within your comfort zone is not enough to make a difference, so you may have to either take on large liabilities (bank loans) or get investors who will share control of your company.

But there is a different story with reaching for fast growth with a sales-driven approach. Which is that everything about the structure and environment of your organization will fundamentally change. First, it will change from an engineering-driven company to a sales-driven one. This means, that no longer will engineers call the shots about the direction and delivery of the software. The sales people will do that and the engineering staff will have to follow.

The reason is that salespeople are expensive and they only go after large customers. Every potential customer can bring in revenue from tens of thousands to millions of dollars. Let us say that a customer is willing to pay a site license that is equal to 25% your current revenue, but wants to add three non-trivial features that is not on the development road map. You have a choice to make here. If you decide that the integrity of your application is more important, then not only will you be writing off a huge dollar amount, but you will anger the salesperson who will probably quit. Which is not what you set out to do.

So, you take on new customers and do further customizations and changes that take the product away from its initial vision. This is not necessarily a bad thing from a business perspective, because it may meet your business goals. But it definitely will establish a new, different pecking order in your organization. And probably some of your technical employees (especially the earliest ones) will leave the organization because their personal growth needs do not coincide with those of the organization.

In a perfect world, sales and engineering could co-exist happily. But in the real world, money talks and the priorities of the salesforce always come first. Once again, there is nothing wrong in being sales-driven. Higher profits can bring financial benefits and security to everyone in the company and that may be the dream of many of them. Just don’t think that the organization and work environment will stay the same once you change your focus.

One differentiating feature on StackExchange is the use of OpenID for registration. As a technical user, I like OpenID, because I don’t have to keep remembering the password for every single site I visit. A few clicks and I am logged in. What could be easier and more convenient than that? Everyone should use OpenID, right?

Not so fast. Here are some lessons from our experiences with talking to users who tried to use the site.

First, people have no idea what OpenID is. So they are confused and skip the registration step. People would feel less lost if you tell them, “Login with your Google, Yahoo! or AOL account” (and whatever other sites support OpenID). It won’t hurt to leave out the word “OpenID” itself.

StackExchange’s statement “Log in with OpenID” is more technically accurate than “Register with OpenID“, but it baffles users because they don’t understand how to register on the site. They don’t realize that a new account is created when they login with an OpenID account.

The standard expectation for users visiting a website is that they want to see a Login form (User name, Password and a Submit button), a “Remember Me” checkbox,  ”Forgot Password” link and a “Register” link. The last takes them to a page where they can submit their information including an email for validation purposes. There is a lot of friction, but this is what users expect. When they see something different, they get antsy and don’t know what to do.

Users have been participating on the site (answering questions) without registering on the site, despite the fact that the OpenID link is available on every answer form. StackExchange (apparently) does not provide the ability to force users to register before they can answer questions.

I don’t have any data for this thought, but I also wonder if users who click through for Google’s OpenID authentication get spooked and wonder if they are allowing another site access to their Google account, which is not true. In the traditional registration mechanism, there is a clean separation between one site and the other and so this issue never comes into the picture.

From an administration perspective, there is another problem. Users who login through OpenID and do not update their profile show up as “unknown (google)”. We don’t have their name, email address or any other information as a means to contact them in the future. Strangely, the system captures more information from the unregistered users who answer questions.

In conclusion, OpenID is good in theory, but may be a little premature for non-technical audiences.

An excerpt:

Although scientifically purer, it often doesn’t make sense to change a single variable at a time. Theoretically, you shouldn’t change the price of your product, your discounting strategy and the types of bundle that you sell, all at the same time. But practically, it can be the right thing to do. It’s more useful to fix the problem than to understand why it’s broken. When a scientist goes on a blind date that doesn’t work out then, in theory, he should fix one variable at a time, and re-run the date. First, he should change the partner but go to the same film and buy the same flowers. Next, he should keep the partner the same, vary the film and keep the flowers the same, and so forth. But the pragmatist in him will, or should, change the girl, the film, the flowers, and buy some new clothes and shave too. If it works, he might not understand why, but at least he’ll have a girlfriend.

Complement this with some reading on basic microeconomics.

In this simplified example above you switched from 3 lines of easily understandable code to something that requires code in the global.asax, another couple classes, and an interface. The end result contains an extra line or two of code, but now the whole thing is “reusable” and can be unit tested. I happen to agree with Joel’s quote which you openly mocked… it *is* more convoluted. It *is* harder to read and grok. And this was an overly simple example… I’m sure it gets more convoluted with large projects.

I am going to ignore IoC for a minute and take this in a general context. Some junior programmers have a tendency to write lengthy code instead of splitting the code into different classes and methods. The code, in itself, is not bad. I am not talking about poor variable names, bad formatting, copy-and-paste or low level silliness. But it is structurally deficient in the sense that

1. Each method will have many lines of code doing different things. In other words, the method has multiple responsibilities.
2. The coding is algorithmic, i.e., although they are coding in an object-oriented language, the coding is very linear in terms of execution.

I used to have a different perspective about people who did this kind of coding, but over time, I have concluded that this has got to do with the (learnable) ability of programmers to deal with complex systems consisting of different building blocks.

Junior programmers need to see, feel and know the entire program. They feel lost and confused otherwise. This is only possible if the code is one place and it has a very linear structure. This is enough for simple programs. But more complex programs require more code.

Well, it happens that a tree-like structure of code is also easy to understand because it is a easy progression from a linear code structure. You start from the root (the main method), navigate linearly, traverse each branch (if you come upon one), re-trace your steps and keep going until you are done. If the tree structure is not deep, you don’t have to keep too many things in your head and the complexity is managed.

The tree-structure is achieved by moving chunks of code into different sub-routines. But here is where the problem starts. You don’t get a true tree structure because you can call different sub-routines from different parts of your program – almost like having a leaf shared by different branches. Now, this is not a problem when the methods are very simple, but when they start manipulating data, it starts getting complex again. Another problem is that in modern event-driven programming environments, there is no “root” of the program. The linearity has been broken. What you have is a graph that has many inter-connections (or dependencies). Once again, complexity rears its ugly head.

We have different techniques to manage this complexity. Object-oriented design (especially encapsulation), refactoring techniques and design patterns help us manage the increasing complexity of software applications. But some of the techniques seem counter-intuitive if you have never used them.

For example, a junior programmer can reduce some complexity in his code by isolating similar code and moving it into a method with parameters. This reduces the code and decreases the complexity. On the other hand, moving data and code into a new class seems to increase code. It seems to imply more code to understand and worry about.

But, of course, this is not true. The point is to move code into a place where it can be completely forgotten. You create the new class that can be tested on its own and then only worry about the code that interacts with it. To a large extent, we already do this by tapping into the framework classes and APIs.

I want to emphasize that this is a learnable skill. These are all well-documented techniques waiting for some investment of time and effort. As you learn them and use them more in your code, you will become familiar with how they work. Some of these are easier and/or more commonly used than others, so it may take a while to be an expert in them all. The point is to keep learning and keep becoming a better programmer.

I have written about this before:

if you are a producer, how do you profit by giving away your product for free? At the moment you stop charging, you lose the revenue from product sales. So you only benefit by making more money via other means than you have lost by forgoing direct sales.

JetBrains may be betting on the scenario that there will be a greater adoption of the free product over Eclipse, which, in turn, drives the sales of the commercial product upwards. This may have been more likely if only the Community Edition had come without many features disabled.

It gets mind-boggling as you go down the comparison list. Almost all the features that would entice someone to ditch Eclipse for IDEA have been removed. Remember people, the biggest problem in Eclipse is not the editor, it is the senseless ability  to configure anything and everything that makes it so difficult to do something simple. So why in the world would you remove (from IDEA) debugging, language support for HTML and environment support for Spring and Hibernate, not to mention any application server? (The blog post mentions debugging, but the feature list has it as missing.)

It almost seems that there was an internal argument inside the company with some people arguing for open source and the other side worrying about cannibalization of sales. Finally, they struck a compromise by releasing a barebones version as open source, which doesn’t help either side of the argument.

The other possibility for JetBrains is that IDEA will attract developers working on Eclipse. Once again, with many important features removed, why should anyone bother? Re-invent the wheels that both Eclipse and IDEA (in the proprietary version) have already invented? I just cannot see it.

Cedric has similar thoughts:

I still see the move from commercial to open source as a sign that the business is struggling. A lot of companies have gone down that path in the past and all of them have tried to make it pass as a selfless action meant to help the community, but the truth is that they were just having a harder time selling their software, so making it open source is usually a last ditch effort to regain mindshare while trying to make money somewhere else.

I can’t think of a single example where a struggling commercial software suddenly started regaining market share when they went open source. Can you? [...]

Whatever side of the fence you stand on, one thing is clear about this move: it means less revenue for JetBrains for the foreseeable future. And what this means is that they will have less means to compete against Eclipse and less power to add features to either of the editions (the Community one or the Ultimate one).

And this is where a lot of companies make a fatal mistake: they think that making their software open source will automatically generate a ground swell of patches and additions from the community that will float them back to the top.

And in my experience, this never happens.

But why am I asking the question in the first place?

Well, much discussion in the technical blogs (such as the NOOP.NL Top 200) is centered around the idea about whether a mediocre programmer will land on a blog where somebody is talking about adopting a simpler approach to programming and then use that idea to stop improving himself and continue writing bad code.

It is a useless subject to argue about because bad programmers do not haunt technical blogs. This is how such a person works:

1. Come in at 9 am or whatever time you are supposed to come in.
2. Find out what task to do.
3. Is the task something that looks familiar? If so, go to Step 7.
4. Interrupt someone in the office. Do they know the answer? If so, go to Step 7.
5. Google for the answer. Is there anything there that works? If so, go to Step 7.
6. Complain to the project manager that the task cannot be done and get something that can be done.
7. Do the task.
8. Is it time to leave? If so, go to Step 11.
9. Is there any other task? If so, go to Step 3
10. Kill time until it is time to leave.
11. Leave.
12. Enjoy the rest of the day.

What they don’t do is spend time visiting technical websites or blogs unless they happen to be landing on them via Google. And those Google visits are only for the purpose of finding some code that works. And issues like intellectual property rights be thrown to the winds.

So let us be clear about the audience when we are arguing about technical debt. We are having an argument among people who know how to write good code and how to test it properly. Some of them use a few more tools and a little more process than the rest, who are constrained either by their lack of knowledge, awareness or business situation to imitate them. No one among this crowd is writing bad code, as in copy-and-paste coding or bad variable names.

Therefore what we are really discussing about and what needs to be answered by people are:

1. You are a good developer who gets things done without Process X or Tool Y.
2. How much beneficial for you would it be to use X or Y?
3. How much time, effort and money do you need to invest for using X or Y to get started in and then be proficient?
4. What is the return on investment if you use X or Y?
5. Let’s understand if X or Y makes sense in your situation, including timelines, budgets, technical environment.

There are literally thousands of programming tools, techniques and processes out there, all claiming to increase productivity. So whatever the “best thing” you are promoting is, you have to justify it in these terms to the good programmers. And someone who cannot justify that is simply selling snake oil.

As for the other programmers, they are out there enjoying the day and definitely not dreaming in code.

The debt metaphor reminds us about the choices we can make with design flaws. The prudent debt to reach a release may not be worth paying down if the interest payments are sufficiently small – such as if it were in a rarely touched part of the code-base.

So the useful distinction isn’t between debt or non-debt, but between prudent and reckless debt. [...]

there [is] a difference between prudent and reckless debt, there’s also a difference between deliberate and inadvertent debt. The prudent debt example is deliberate because the team knows they are taking on a debt, and thus puts some thought as to whether the payoff for an earlier release is greater than the costs of paying it off. A team ignorant of design practices is taking on its reckless debt without even realizing how much hock it’s getting into.

It seems that Martin is suggesting that the best debt you should have is the prudent-inadvertent debt, where you only understand the technical debt after working with the application. It is not a case of recklessness where you decided to ignore design, or be content in your ignorance. Or a case where you weighed all the pros and cons and then decided for the quick-and-dirty approach. The prudent-inadvertent debt is something that happened despite your best intentions and efforts.

The way to explain this is that there are potentially different ways to design an application. For example, if you take the creational design patterns, you may have to choose between competing patterns. You may start using a Factory pattern, but then need to change that for something more flexible. Or you start out with a relational database design and then later find you have to denormalize or use a NoSQL database because the application became too successful. It is hard to predict the changes in requirements and sometimes, you have to make the decision using present day considerations.

There are always gray areas in those quadrants. Assume that you are building an online application. It is impossible to know ahead of time whether you will get 100 visitors per day or 10,000 or a million. You could choose to target a million, but that could be over-engineering the application. Maybe 1,000 seems a happy medium or maybe you are making a terrible mistake of under-estimating traffic. Making design decisions in these cases can seem reckless or prudent after the fact.

Perhaps the real mistake is to think of design as an independent process. Much is tied to business decisions and forecasts. And where your technical debt falls is determined by those business factors.

[...] many people I care about [...] made a healthy living off of building “line of business” applications in FoxPro, which Microsoft has now officially shut down. For those who did Office applications, Visual Basic for Applications has now been officially deprecated in favor of VSTO (Visual Studio Tools for Office), a set of libraries that are available for use by any .NET application language, and of course classic Visual Basic itself has been “brought into the fold” by making it a fully-fledged object-oriented language complete with XML literals and LINQ query capabilities.

Which means, if somebody working for a small school district in western Pennsylvania wants to build a simple application for tracking students’ attendance (rather than tracking it on paper anymore), what do they do?

I am in two minds about this. In agreement with Ted, it would be nice to crank out quick business applications using simple tools. In fact, many applications that are built using .NET or Java could be automated using Excel or Access, not to say even a Word document that is put in a shared folder! It is one of the characteristics of human beings that they will spend more money than they need for things that they could have lived without.

The problem, unfortunately, is that people also display the following characteristics:

• They get new ideas, i.e., when they see what software can do, they want it to do more.
• They change their mind about how things should work.
• They are not organized: they need the software to protect them from their own mistakes or provide them more information to help them out.
• They don’t understand the millions of man hours that have gone into fine-tuning the functionality in sophisticated off-the-shelf products and demand that in their own applications.

This is all apart from external business reasons for application changes. So there is no “simple” application because the moment it is built, people want to change it.

For the developer, there are 2 choices: Walk away after building it, or continue to make changes for the customer. But even in the first chance, there may be a warranty/guarantee period where the developer continues to be involved with the product. And these changes become more and more difficult with the tools that we had in the past.

There are many dimensions where the “simple” tools of the past made it more difficult. Consider deployment – it is infinitely easier to manage deployment on one web server than worry about many different Windows (or DOS!) machines. Consider simplicity – plain JavaScript versus JQuery. Consider separation of concerns in different architectural layers instead of code lying in the different layers, including the “business” layer and triggers.

There are many more tools today to learn, but they make it simpler, not harder, to code. Ted has a point that the standalone developer has to go through several weeks of learning. But if we are talking about college students, those 6 weeks is spread across a few years of study – not a big deal. But even if it is a person making a new career move into programming, there has to be some investment in educating oneself. Otherwise the programmer will end up fielding more and more support calls instead of moving from one project to another.

Also,  it is possible to build smaller applications using a limited set of present-day technologies. If you look at the 5-week list that Ted put up, it is plausible that a good programmer can write a small application without using or knowing much of them. For example, Joel Spolsky and Co. probably don’t use many of the tools in the .NET space or at least they hadn’t been using them during the Wasabi Era.

Finally, I don’t see it necessarily as a disaster that people have to rewrite software once in a while. Sometimes, a business can only afford to spend so much money building an application. If the business grows, maybe they will throw out what they have to and get something better. Just as they would buy a larger conference table or bigger coffee maker after throwing what they have into the dumpster. Sometimes, the smaller investment helps them get to the next level, where they have to spend more money.

Much of the recent debate over unit tests and Inversion of Control has been the meta discussion: “Unit tests are hard” versus “No, they are not” and so on. Which is why Rob Conery’s post on IoC and its sequel are worth reading. Well, at least for the .NET audience. Honorable mention also goes to Justin Etheredge for a similar (but less detailed) article.

I would note a few points. First is that not all IDEs and IoC containers are created equal. Those who are working outside the .NET space and are not using commercial products can sometimes find the going tough. Maybe this is just my experience with open source tools, but it does take a non-trivial amount of time to set up your environment properly and then work with it easily.

This should not deter someone who wants to get it done. But if someone is exploring how it is working, there is a barrier for such beginners and can sour them on the experience. My advice to the open source community is to come up with installation packages that can make it easy for such programmers to get started with these tools easily. Greater usability will mean proliferation of best practices in the general programming community.

Second, there is a virtuous cycle in place with these programming practices. Someone using TDD has a greater chance of using IoC. And that same person is more likely to learn and use design patterns where applicable. So you perhaps only have to give a small push to a programmer to get them rolling on the way to adopt these best practices. Start them with unit tests and soon they will be on to bigger things.

Finally, while we have good tools, there is still scope for improvement. A common complaint about TDD are tests that achieve greater coverage, but are either meaningless or boiler-plate stuff. There can be more intelligent unit testing tools that generate generic tests, allow the programmer to override them where necessary and spend more time on writing tests for more complex code. We are getting there, especially in the .NET space, but more of that would be better.