Coding Horror

Whitespace: The Silent Killer

Jeff Atwood — Mon, 09 Nov 2009 00:34:09 -0800

Ever have one of those days where everything you check into source control is wrong?

Also, how exactly is that day is different from any other? But seriously.

Code that is visible is code that can be wrong. No surprise there. But did you know that even the code you can't see may be wrong, too?

These are the questions that drive young programmers to madness. Take this perfectly innocent code, for example.

Looks fine, doesn't it? But hold on. Wait a second. Let's take another, closer look.

OH. MY. GOD!

If you're not a programmer, you may be looking at these two images and wondering what the big deal is. That's fine. But I humbly submit that, well, you're not one of us. You don't appreciate what it's like to spend every freaking minute of every freaking day agonizing over the tiniest details of the programs you write. Not because we want to, you understand, but because the world explodes when we don't.

I mean that literally. Well, almost. If one semicolon is out of place, everything goes sideways. That's how programming works. It's fun! Sometimes! I swear!

We got into this industry because, quite frankly, we are control freaks. It's who we are. It's what we do. Now to imagine, to our dismay, that there's all this stupid, useless whitespace at the ends of our lines. Stuff that's there, but we can't see it. Well, those are the nightmares OCD horror movies are made of. I have a full-body itchiness just talking about it.

Depending on how far down the rabbit-hole you want to go, there's any number of things you could do here:

Have a post-build step, perhaps something with a regular expression like \s*?$ in it, that auto-cleans extra spaces checked into source control
Execute a local macro which removes whitespace from ends of lines
Have a special rule to highlight extra spaces
Run your IDE in whitespace-always-visible mode, or toggle it frequently

OK, fine, so maybe the world won't explode if there are a few extra bits of whitespace in my code.

But all the same, I think I'll go back and make extra double plus sure no more of that pesky whitespace has accumulated in my code when I wasn't looking. Just because I can't see it doesn't mean it's not out to get me.

[advertisement] JIRA 4 - Simplify issue tracking for everyone involved. Get started from $10 for 10 users.

Preserving Our Digital Pre-History

Jeff Atwood — Thu, 05 Nov 2009 04:33:10 -0800

I've spent a significant part of my life online. Not just on the internet, I mean, but on modems and early, primitive online communities. Today's internet is everything we couldn't have possibly dared to imagine twenty-five years ago, but there is a real risk of these early, tentative digital artifacts -- and for some, the beginnings of our Hacker Odyssey -- being lost forever in the relentless deluge of online progress. Sure, every single thing that happened in 2004 is documented exhaustively online. But 1994? 1984? Not so much.

That's where Jason Scott comes in.

You may know Jason Scott from BBS The Documentary. Or, perhaps you're familiar with textfiles.com, his massive (and growing) archive of what passed for blogs and forums in the earliest online era.

A wonderful thing happened in the 1980s: Life started to go online. And as the world continues this trend, everyone finding themselves drawn online should know what happened before, to see where it all really started to come together and to know what went on, before it's forgotten.
When a historian or reporter tries to capture the feelings and themes that proliferated through the BBS Scene of the early 1980's, the reader nearly always experiences a mere glimpse of what went on. This is probably true of most any third-party reporting, but when the culture is your own, and when the experiences were your own, the gap between story and reality is that much wider, and it's that much harder to sit back and let the cliche-filled summary become "The Way It Was." You want to do something, anything so that the people who stumble onto the part of history that was yours know what it was like to grow up through it, to meet the people you did, to do the things you enjoyed doing. Maybe, you hope, they might even see the broader picture and the conclusions that you yourself couldn't see at the time. This is history the way the chronicled want it to be.

Jason is nothing less than our generation's digital historian in residence. When GeoCities went permanently offline a week ago, he was there to help preserve it for posterity.

BBS: The Documentary was a major milestone in his ongoing effort to document our digital pre-history. But it's only the beginning; there's also a huge documentary on text adventures, Get Lamp, that's been in the works for a few years now. Unfortunately, progress has been slow. Because while being a digital historian is great, it's not exactly something you get paid to do.

But maybe we can change that. Witness Jason's kickstarter proposal:

Throughout all this, I had a day job - computer administration. It paid well, but I paid for it with my health. When my most recent employer and I parted ways, I decided I'd take this time finish some of the bigger projects I've been working on.
I suddenly thought back to Kickstarter and got this crazy idea - what if I simply asked the world and fans to contribute a bit of money towards keeping me somewhat solvent, and give me the opportunity to go full-time with computer history? If I was able to get all these things done over the years, what if I just asked people to subscribe or give me some patronage and in return I fill their free time with cool stuff to look at, learn from, and enjoy?

There are so many people whose online presences I greatly admire. But very few of them will go on to become part of the permanent written history of this era. I have no doubt whatsoever that Jason Scott is one of those people who will, thanks to his tireless efforts to preserve the flotsam and jetsam of our digital past, stuff that would otherwise be overlooked by the mainstream and lost forever.

I've pledged $100. It is an honor to support his ongoing work of preserving our shared digital pre-history. His history, is my history, is our history. A history of geeks, dorks, dweebs, nerds, and generally computer-obsessed misfits, but nonetheless -- it's something we all share.

If this is something you believe in, I urge you to pledge as well.

[advertisement] JIRA 4 - Simplify issue tracking for everyone involved. Get started from $10 for 10 users.

Stack Overflow Careers: Amplifying Your Awesome

Jeff Atwood — Tue, 03 Nov 2009 01:07:43 -0800

That Stack Overflow thing we launched a year ago? It's been going pretty well so far.

Of course, everyone knows you could code Stack Overflow in a long weekend. It's trivial. Assembling a worldwide community of smart, engaged software developers? That's a whole different ball of wax. Stack Overflow is a site by programmers, for programmers; it's only as good as the programmers who choose to participate.

Stack Overflow isn't about me. Or anybody else on the Stack Overflow team for that matter.
Stack Overflow is you.
This is the scary part, the great leap of faith that Stack Overflow is predicated on: trusting your fellow programmers. The programmers who choose to participate in Stack Overflow are the "secret sauce" that makes it work. You are the reason I continue to believe in developer community as the greatest source of learning and growth. You are the reason I continue to get so many positive emails and testimonials about Stack Overflow. I can't take credit for that. But you can.
I learned the collective power of my fellow programmers long ago writing on Coding Horror. The community is far, far smarter than I will ever be. All I can ask – all any of us can ask – is to help each other along the path.

I am continually humbled by the skill and expertise of the programmers who volunteer time to Stack Overflow. These programmers graciously donate tiny slivers of their day to help us -- and themselves -- become better programmers. These 5 and 10 minute slices of effort, across hundreds of thousands of questions and answers, become a permanently archived (and creative commons wiki licensed) bread crumb content trail for future programmers to follow, edit, and contribute to themselves over time.

I'm thrilled to see Stack Overflow working so well for both askers and answerers; the "pay it forward" model of programmers helping their peers is exactly what we were shooting for. We'll never change the world, but it sure is nice to be able to improve our small corner of it just a little bit. Remember: bad code that isn't written, is bad code that another poor programmer won't have to debug. If we don't reach out to ~~slap~~help new programmers and teach them the lessons we learned the hard way, who will? I'm only exaggerating a little when I say that the future of our entire profession depends on it.

If you're actively participating on Stack Overflow, we now have another way to convert those slices of effort into something that actively furthers your professional goals – Stack Overflow Careers.

What is careers.stackoverflow.com? It's a few things:

a completely free, public CV hosting service for programmers, to share the cool stuff you've coded and created with the world.
a way to explicitly link your Stack Overflow profile with your CV, to provide concrete examples of your communication skills and individual expertise to anyone who is interested.
a better way to connect great programmers with the best programming jobs, for those who opt into the small annual listing fee.

In short, Stack Overflow Careers amplifies your awesome.

I won't lie to you. This is also a business. That's why there are nominal opt-in listing fees for those programmers interested in seeking employment, and substantial fees for hiring managers who want to tap into the smart developers who grok Stack Overflow.

update: I apologize if I wasn't clear. It is 100% free, forever, to create a public CV, put whatever HTML content you want in it, and link it to your Stack Overflow profile. Like so:

These are of course freely indexable and searchable on the web.

Beyond the free public component, there is a private (and completely optional) subscription component. For those programmers actively seeking employment, a small annual subscription fee allows inclusion in a private employer search UI. This is also explained in the faq and about.

That said, we're also trying to do something a bit different here. Something better than the endless, mind-numbing acronym sea of monster.com, dice.com, et al. Joel and I believe current hiring practices for programmers are incredibly broken. We think we can do better.

We love our work, and so should you. Our goal isn't to put warm bodies in front of interviewers. Our goal is to create love connections. Instead of avid programmers pursuing disinterested and distracted companies, it's the other way around -- savvy companies who understand the competitive advantages of having the best programmers will pursue you. We connect smart, engaged hiring managers who "get it" with top programmers who love to code.

If you love to code, too, I encourage you to create your own Stack Overflow CV. Keep it private, or make it public via the URL of your choice -- it's completely free either way. If you think you might be actively looking for a job in the next 3 years, take advantage of our outrageously low promotional pricing of $29 for a 3 year filing. That way, at any point in those 3 years, you can flip a switch and become visible to hiring managers. Or not. It's totally up to you.

(also, if you're hiring, and your company appreciates top software engineers -- and you think you can convince our tough audience of that -- email us)

[advertisement] JIRA 4 - Simplify issue tracking for everyone involved. Get started from $10 for 10 users.

Revisiting "The Fold"

Jeff Atwood — Mon, 26 Oct 2009 15:25:05 -0800

After I posted my blog entry on Treating User Myopia I got a lot of advice. Some useful, some not so useful. But the one bit of advice I hadn't anticipated was that we were not making good use of the area "above the fold". This surprised me. Does the fold still matter?

The fold refers to the border at the bottom of the browser window at the user's default screen resolution. Like so:

Way back in the dark ages of 1996, it was commonly thought that users didn't know how to scroll a web page.

On the Web, the inverted pyramid becomes even more important since we know from several user studies that users don't scroll, so they will very frequently be left to read only the top part of an article.

Thus, it was critically important to cram in as much content in as possible above that fold, as anything below it was invisible to a huge number of users. They didn't know how to scroll, so they would never find it. Jacob Neilsen, renowned usability expert, is the author of the above quote. But he recanted his position in 2003:

In 1996, I said that "users don't scroll." This was true at the time: many, if not most, users only looked at the visible part of the page and rarely scrolled below the fold. The evolution of the Web has changed this conclusion. As users got more experience with scrolling pages, many of them started scrolling.

Scrolling is an example usability versus learnability. It was always my belief that users quickly learned to scroll, otherwise they were permanently crippled as web citizens. If you can't learn to scroll within an hour or so of using the web, you're going to have an awfully stunted experience -- so much so that you're probably better off not using it at all. In short, if you use the web, you know how to scroll, almost by definition. It is a fundamental skill.

Even today, people will cite the ancient, irrelevant rule of The Fold as if it's still law. In fact, I was just talking to a friend of mine who expressed his frustration at dealing with a middle manager who was using the "content must be above the fold" rule as a weapon, and demanding that all page content appear above the fold. It's terribly misguided.

Although thoroughly debunked, there are still some hidden dangers from the fold, and subtlety to how users react to it. As documented by a recent usability study on the fold, there are three specific pitfalls to watch out for:

Don't cram everything in above the fold. Users will explore and find your content -- as long as the page "looks" scrollable.
Watch out for stark, horizontal lines that happen to line up with the fold. This is the only factor that causes users to stop scrolling, because the page looks done and complete. Instead, have a small amount of content just visible, poking up above the fold. This encourages scrolling.
Avoid in-page scroll bars. The standard browser scrollbar is an indicator of the amount of content on the page that users learn to rely on. Placing and other elements with scroll bars on the page can break this convention -- and may lead to users not scrolling.

These are excellent guidelines, backed by actual eye tracking and experimental results. You know, science! But how do they apply to me? First, I established where the fold actually was. Per Google Analytics, about 25% of our users are using screen resolutions where the page fold is at about 700 or 800 pixels of height. And remember, browsers have a lot of horizontal chrome that tends to squander that height -- toolbars, status bars, tabs, etcetera. The fold is probably much closer than you think it is.

Next, I looked at the advice I had been given regarding the top of the page. Sure enough, we had a bunch of irrelevant UI at the top that didn't really matter: things like redundant page titles, and two line title entry. We were wasting critical real estate at the top of the page! For the 25% of users who have a 700 or 800 pixel fold, items were pushed down far enough that they might not actually be visible. Worse still, the strong bottom border of the text entry area with the drag slider could possibly align with the page fold itself -- leading the user to believe that nothing is below there and failing to scroll.

It's not only a basic rule of writing, it's also a basic rule of the web: put the most important content at as close to the top of the page as you can. This isn't new advice, but it's so important that it never hurts to revisit it periodically in your own designs.

In treating user myopia, it's not enough to place important stuff directly in the user's eyepoint. You also need to ensure that you've placed the absolute most important stuff at the top of the page -- and haven't created any accidental barriers to scrolling, so they can find the rest of it. The fold is far less important than it used to be, but it isn't quite as mythical as Bigfoot and the Loch Ness Monster quite yet.

[advertisement] JIRA 4 - Simplify issue tracking for everyone involved. Get started from $10 for 10 users.

Treating User Myopia

Jeff Atwood — Thu, 22 Oct 2009 23:59:59 -0800

I try not to talk too much about the trilogy here, because there's a whole other blog for that stuff. But some of the lessons I've learned in the last year while working on them really put into bold relief some of my earlier blog entries on usability and user behavior.

One entry in particular that I keep coming back to is Teaching Users to Read. That was specific to dialog boxes, which not only stop the proceedings with idiocy, but are their own delightful brand of user interface poison. Fortunately, you don't see dialogs in web apps much, but this sort of modal dialog lunacy is, sadly, becoming more popular in today's AJAX-y world of web 2.5. Those who can't learn from history are doomed to repeat it, I guess.

Having five more years of development experience under my belt, I no longer believe that classic Larson strip is specific to dialog boxes.

The plain fact is users will not read anything you put on the screen.

What we're doing with the trilogy is not exactly rocket surgery. At its core, we run Q&A websites. And the most basic operation of any Q&A website is … asking a question. Something any two year old child knows how to do.

When we launched superuser.com, that was our ~~third~~fourth Q&A website. This one is for power users, and it's the broadest to date, topic-wise: anything dealing with computer software or hardware (that isn't gaming) is allowed.

We've been at this for over a year now, doing nothing but relentlessly polishing and improving our Q&A engine based on community feedback. We're not particularly good, but we do try very, very hard not to suck. I thought surely, surely we must have something as simple as the ask question form down by now.

How foolish I was.

Let's take a look at one recent superuser question. I'm presenting it here as it would have been seen by the user who asked the question, while they were entering it on the ask question form.

Immediately, there's a problem. The question formatting is completely wrong! It's one big jumble of text.

Our formatting rules aren't complicated. You can get a lot done with a bunch of simple paragraphs. We use Markdown, which offers basic formatting conventions that ape ASCII conventions. On top of that, we offer a real-time preview of how your question will look once submitted, directly under the question entry area. But none of that seemed to work for this particular asker, who, apparently, was totally satisfied with obviously broken formatting -- even though a few choice carriage returns would have worked wonders, and been immediately visible in the live preview.

Yes, yes, it inevitably gets whipped into shape through the collective efforts of our legions of community editors -- but that's not the point. It's best if the original asker gets the question formatted right to start with, and it is our job as UI designers to make that outcome as statistically likely as we can.

To that end, we've put a bunch of helpful tools on the ask question page to help users get the formatting right. As UI designers, here's how we see the ask question page:

We've provided a toolbar with a neon pink help button above the question body, and to the right of the question body, we've provided a handy formatting quick reference with a link to the full formatting reference (which opens in a tab / new window by default).

But none of that matters, because here's how the user sees the ask question page:

Or rather, here's everything the user doesn't see.

When I said users don't read anything you put on the screen, I was lying. Users do read. But users will only read the absolute minimum amount of text on the screen necessary to complete their task. I can't quite explain it, but this kind of user myopia is epidemic. It's the same problem, everywhere I turn.

How do we treat user myopia? How do we reach these users? The ask question page is already dangerously close to cluttered with helpful tips, but apparently these helpful buttons, links, and text are all but invisible to a large segment of the user population. Sure, you could argue that Super User tends to attract less sophisticated users, but I see the exact same problem with programmers on Stack Overflow. As new users, a significant percentage of them can't figure out how to format code, even though there's not only a toolbar button that does it for you, but help text on the right explicitly describing how to do it manually. (Just indent 4 spaces. Spoiler alert!)

More and more, I'm thinking we need to put the formatting help -- for new users only -- directly in their line of sight. That is, pre-populate the question entry area with some example formatting that is typical of the average question. Nothing complicated. But at least then it'd be in the one -- and apparently the only one -- place myopic users are willing to look. Right in front of their freakin' faces.

The next time you're designing a UI, consider user myopia. You might be surprised just how myopic your users can be. Think long and hard about placing things directly in front of them, where they are not just visible, but unavoidable. Otherwise they might not be seen at all.

[advertisement] JIRA 4 - Simplify issue tracking for everyone involved. Get started from $10 for 10 users.

The Interview With The Programmer

Jeff Atwood — Sun, 18 Oct 2009 22:00:00 -0800

If the internet has perfected anything, it's the art of the crappy, phoned-in, half-assed email "interview". For all those who have bemoaned the often pathetic state of internet journalism, when it comes to interviews, you're largely correct. The purpose of most of these interviews is quick and dirty content filler with semi-famous folk spouting off whatever random thoughts they happen to have in their head at that exact moment. The Nixon Interviews, it ain't.

That's why I'm normally not a huge fan of interview books, because interviews take an enormous amount of time and an enormous amount of legitimate, skilled journalistic effort to get right. Almost nobody does.

Imagine my surprise when Coders at Work: Reflections on the Craft of Programming turns out to be that wonderfully rare intersection of uncommonly skilled interviewing and 15 of the most influential programmers to ever touch a keyboard.

Yes, this is the same book Joel recently recommended in his controversial Duct Tape Programmer entry, which is why I was all the more skeptical. But he's dead on. I could (and probably will, knowing me) fill a year worth of blog posts just with the thought provoking quotes and two-paragraph insights revealed in these interviews. It's astonishingly good. If, after reading what these brilliant programmers have to say, you aren't motivated to research some programming topic mentioned inside, pack it in, because you aren't even trying any more.

I also realized Coders at Work can potentially serve as a job interview filter. If the next programmer you interview can't identify at least one of the programmers interviewed in Coders at Work and tell you roughly what they're famous for …

Frances Allen	Joe Armstrong	Joshua Bloch
Bernie Cosell	Douglas Crockford	L. Peter Deutsch
Brendan Eich	Brad Fitzpatrick	Dan Ingalls
Simon Peyton Jones	Donald Knuth	Peter Norvig
Guy Steele	Ken Thompson	Jamie Zawinski

… I'd say that's an immediate no-hire.

Incidentally, I saw the first Stack Overflow user reference on page 265, in the interview with Simon Peyton Jones, who mentions one Norman Ramsey. Hmm, I thought, that name sounds awfully familiar. And indeed it was!

I would be remiss if I did not mention that the author, Peter Seibel, was directly inspired by Susan Lammers' classic 1986 book Programmers at Work: Interviews With 19 Programmers Who Shaped the Computer Industry.

This is one of my absolute favorite musty old computer books for many of the same reasons. As sources of inspiration go, this one is particularly … er, inspired. Programmers at Work isn't just the archetypal programmer interview book -- it also holds up amazingly well for a book that is over twenty years old. It is a testament to the timelessness of not just code, but the art of coding, as exemplified by these 19 programmers. I believe Peter has legitimately crafted a modern remake that will be relevant for another twenty years. And I hope I don't have to tell you how extraordinarily rare that is among technical books.

(Some -- but not all from what I can tell -- key interviews from Programmers at Work were placed online last year by the author. So if you want to get a flavor of the book, check it out.)

Another notable recent collection of interviews is Masterminds of Programming: Conversations with the Creators of Major Programming Languages.

Although I definitely enjoyed this book, there's something about the focus on programming languages and interview style that didn't quite grab me as forcefully as Coders at Work did. Also, if we're going to do languages, I'd like to see a bit broader representation -- perhaps a Volume II with Smalltalk, Ada, Pascal and so on?

These books are a potent reminder that computers are mostly a reflection of the people using them. In the art of software development, studying code isn't enough: you have to study the people behind the software, too.

[advertisement] JIRA 4 - Simplify issue tracking for everyone involved. Get started from $10 for 10 users.

The State of Solid State Hard Drives

Jeff Atwood — Tue, 13 Oct 2009 23:59:59 -0800

I've seen a lot of people play The Computer Performance Shell Game poorly. They overinvest in a fancy CPU, while pairing it with limited memory, a plain jane hard drive, or a generic video card. For most users, that fire-breathing quad-core CPU is sitting around twiddling its virtual thumbs most of the time. Computer performance is typically limited by the slowest part in your system. You'd get better overall performance by building a balanced system and removing bottlenecks.

One of those key bottlenecks -- and in my experience, the one typical users are most likely to underestimate -- is the hard drive. I've been a long time advocate of having a small, fast 10,000 RPM boot drive for your OS and key applications, and a larger, slower secondary drive for storage. The two drive approach is a smart strategy, regardless of your budget.

Hard drives may not be particularly sexy bits of hardware, but we're on the verge of a major transition point in storage hardware -- from physical, magnetic platters to solid state memory. I was an early solid state (SSD) drive adopter with my last laptop purchase, and it was a profound disappointment. Those first and second generation SSD drives turned out to be slower than their magentic equivalents, despite the eager promises of vendors. On top of that, they were incredibly expensive, and of limited capacity. Running Windows Vista on an early 32 gigabyte SSD was an exercise in pain and frustration on so many levels. What's not to love? A lot.

I eventually sold that SSD and replaced it with a traditional hard drive. I had grown deeply disillusioned with SSD drives. That is, until I read Linus Torvalds' report on the Intel X-25 SSD:

I can't recall the last time that a new tech toy I got made such a dramatic difference in performance and just plain usability of a machine of mine.
The whole thing just rocks. Everything performs well. You can put that disk in a machine, and suddenly you almost don't even need to care whether things were in your page cache or not. Firefox starts up pretty much as snappily in the cold-cache case as it does hot-cache. You can do package installation and big untars, and you don't even notice it, because your desktop doesn't get laggy or anything.
So here's the deal: right now, don't buy any other SSD than the Intel ones, because as far as I can tell, all the other ones are pretty much inferior to the much cheaper traditional disks, unless you never do any writes at all (and turn off 'atime', for that matter).

At that moment, SSD drives came of age. And by that I mean, they began to justify their hefty price premium at last. But that was almost a year ago, and even the Intel drives -- as great as they were -- had some teething problems. Not to mention that price and capacity were still ongoing concerns.

But when Intel introduced the second generation, mainstream X25-M drives, that's when I knew SSDs were poised to go mainstream. Now, those drives are still anything but cheap, at $289 for 80 GB and $609 for 160 GB. But they offer more performance than the original X25-E that Linus reviewed, at about half the price, with hardware fixes to address the fragmentation issue that plagued the original model.

Intel was the only game in town for about a year, but fortunately for us consumers, the competition finally caught up. The new Indilinx controller models, such as this Crucial 128 GB SSD, are just as fast as the X25-M. And, best of all, they're cheaper, while also offering a not-insubstantial bump to 128 GB of storage!

I picked this model up for $325 plus tax and shipping. And, frankly, I was blown away by the performance difference compared to the 300 GB Velociraptor I had in my system before. That drive is not exactly chopped liver; it's incredibly fast by magnetic platter drive standards. But it's beyond slow next to the latest SSDs. See for yourself:

This is just an excerpt, browse the reviews for more detail, but I was astonished how often this drive (based on the Indilinx Barefoot controller) topped the charts. Suffice it to say, the performance increase is not subtle. All those little pauses while your system pulls some chunk of data off the hard drive? They simply cease to exist.

How much do I like this drive? I like it so very much I bought one for every member of the Stack Overflow team, as a small gesture of thanks for enduring new feature crunch mode. I couldn't sell my old Velociraptor on eBay fast enough.

In my humble opinion, $200 - $300 for a SSD is easily the most cost effective performance increase you can buy for a computer of anything remotely resembling recent vintage. Whether you prefer the 80 GB X25-M SSD or the 128 GB Crucial SSD, it's money well invested for people like us who are obsessive about how their computer performs.

Trust me, you will feel the performance difference of a modern SSD in day to day computing. That's far more than I can say for most of today's CPU and memory upgrades. The transition from magnetic storage to solid state storage is nothing less than a breakthrough. It's already transformative; I can only imagine how fast, cheap, and large these drives are going to be in a few years. So, if you've ever wondered what performance would be like if everything was in RAM all the time -- well, we just got one giant step closer to that.

[advertisement] JIRA 4 - Simplify issue tracking for everyone involved. Get started from $10 for 10 users.

The Xanadu Dream

Jeff Atwood — Mon, 12 Oct 2009 02:38:33 -0800

Links are the fundamental building blocks of the web. And every time I click on one, I can't help recalling the odd visionary who came up with the original idea of clickable links in text, aka hypertext, in 1963 -- Ted Nelson.

Ted Nelson is, shall we say, a character. He has gone on record many times with the four maxims that guide his life. He isn't shy about sharing them with anyone he meets:

most people are fools
most authority is malignant
God does not exist
everything is wrong

And that, in a nutshell, is pretty much everything you need to know about Ted Nelson. He is the archetypal borderline autistic, non-conformist, free-thinking technologist. Any resemblance between Ted and your average programmer is, I'm sure, completely coincidental. That's why his story is so fascinating to me. It hits close to home.

Like most programmers, Ted's reach often exceeded his grasp. Ted's vision of hypertext was far more grandiose than the motley assortment of links that is today's web. His vision was (and is) a bit of computer history lore that has become the stuff of legend: Project Xanadu.

Xanadu, a global hypertext publishing system, is the longest-running vaporware story in the history of the computer industry. It has been in development for more than 30 years. This long gestation period may not put it in the same category as the Great Wall of China, which was under construction for most of the 16th century and still failed to foil invaders, but, given the relative youth of commercial computing, Xanadu has set a record of futility that will be difficult for other companies to surpass. The fact that Nelson has had only since about 1960 to build his reputation as the king of unsuccessful software development makes Xanadu interesting for another reason: the project's failure (or, viewed more optimistically, its long-delayed success) coincides almost exactly with the birth of hacker culture. Xanadu's manic and highly publicized swerves from triumph to bankruptcy show a side of hackerdom that is as important, perhaps, as tales of billion-dollar companies born in garages.

Among people who consider themselves insiders, Nelson's Xanadu is sometimes treated as a joke, but this is superficial. Nelson's writing and presentations inspired some of the most visionary computer programmers, managers, and executives - including Autodesk Inc. founder John Walker - to pour millions of dollars and years of effort into the project. Xanadu was meant to be a universal library, a worldwide hypertext publishing tool, a system to resolve copyright disputes, and a meritocratic forum for discussion and debate. By putting all information within reach of all people, Xanadu was meant to eliminate scientific ignorance and cure political misunderstandings. And, on the very hackerish assumption that global catastrophes are caused by ignorance, stupidity, and communication failures, Xanadu was supposed to save the world.

The above text is excerpted from the definitive 1995 Wired article on Project Xanadu, which is still as electrifying to read today as it was then. The hubris and sheer scale of the Xanadu dream are at turns both inspiring and desperately, hopelessly out of touch.

Xanadu has 17 rules defining its behavior. As you're reading through this list, ask yourself how many of these rules are satisfied by the current world wide web, even in part:

Every Xanadu server is uniquely and securely identified.
Every Xanadu server can be operated independently or in a network.
Every user is uniquely and securely identified.
Every user can search, retrieve, create and store documents.
Every document can consist of any number of parts each of which may be of any data type.
Every document can contain links of any type including virtual copies ("transclusions") to any other document in the system accessible to its owner.
Links are visible and can be followed from all endpoints.
Permission to link to a document is explicitly granted by the act of publication.
Every document can contain a royalty mechanism at any desired degree of granularity to ensure payment on any portion accessed, including virtual copies ("transclusions") of all or part of the document.
Every document is uniquely and securely identified.
Every document can have secure access controls.
Every document can be rapidly searched, stored and retrieved without user knowledge of where it is physically stored.
Every document is automatically moved to physical storage appropriate to its frequency of access from any given location.
Every document is automatically stored redundantly to maintain availability even in case of a disaster.
Every Xanadu service provider can charge their users at any rate they choose for the storage, retrieval and publishing of documents.
Every transaction is secure and auditable only by the parties to that transaction.
The Xanadu client-server communication protocol is an openly published standard. Third-party software development and integration is encouraged.

It is instructive, then, to consider the primary ways in which the modern web is functionally broken:

Link rot. The odds of a hyperlink working are inversely proportional to the age of that hyperlink. Old links frequently break, because the server hosting the content disappears for any number of reasons over time. I've gotten to the point where I dread clicking on links from old web pages, because the per-click success rate is so abysmally low.
Every website has unique usernames and passwords. There is almost no reliable centralized form of identity on the internet, and those few that do exist are often poisoned by explicit commercial affiliation, such as Facebook Connect and Microsoft Passport. This is why curated anonymous, lightweight participation dominates the net -- best illustrated by Wikipedia.
No redundancy. If content is driven offline by temporary high traffic levels or, worse, catastrophic data loss, there may not be any way to recover that content. I know that Digg has services which auto-mirror highly rated links because Digg traffic can be so toxic to the destination links. (Ironically, all duggmirror.com links redirect to amazon.com now, which illustrates how ephemeral all this stuff tends to be.) I suppose if you're lucky the wayback machine will eventually pick up a historical copy of the content, or the Google cache will hold a copy for some unknown amount of time while the site is offline. You'd probably have better odds praying for missing content to reappear.

Let's put aside the more ambitious parts of Project Xanadu for the moment. The current world wide web does basically one thing: simple, stupid, mindless hyperlinks. But even that alone was enough to build a functional and useful internet for the world. And Google was able to build a zillion dollar algorithm out of discovering the relationship between those dumb hyperlinks.

All that, when the most fundamental building block of the web, the hyperlink, barely works at all. Hyperlinks are fraught with peril and pitfalls even under the best of conditions. The current state of hyperlinking is almost literally the stupidest thing we could build that works. Frankly, the current system sucks beyond belief, as Ted himself notes:

HTML is precisely what we were trying to prevent -- ever-breaking links, links going outward only, quotes you can't follow to their origins, no version management, no rights management.

The next time you're about to embark on a grandiose, glorious software Xanadu Dream of your own, take Dare's advice.

The bottom line is that a lot of the time it's OK to create a solution that solves 80% of the problem. Always remember that shipping is a feature.

Consider the reality of what's actually possible, what people can understand, and what us all too human programmers can practically implement. It might not be the Xanadu you dreamed of -- heck, it might even suck -- but it'll at least have a fighting chance of existing in reality rather than fantasy.

[advertisement] JIRA 4 now available - full featured version starting from $10.

Email: The Variable Reinforcement Machine

Jeff Atwood — Mon, 28 Sep 2009 02:00:32 -0800

How often do you check your email per day?

Does checking your email make you more productive or less productive?

Oh, sure, we delude ourselves into thinking we're being extra-productive by obsessively checking and responding to our email, but in reality we're attending too frequently to our own desire for gratification and sabotaging our own productivity in the process.

As Dan Ariely explains in a postscript to Predictably Irrational, he smells a rat, and so should you:

Skinner distinguished between fixed-ratio schedules of reinforcement and variable-ratio schedules of reinforcement. Under a fixed schedule, a rat received a reward of food after it pressed the lever a fixed number of times -- say 100 times. Under the variable schedule, the rat earned a food pellet after it pressed the lever a random number of times. Sometimes it would receive the food pellet after pressing 10 times, and sometimes after pressing 200 times.
Under the variable schedule of reinforcement, the arrival of the reward is unpredictable. On the face of it, one might expect that the fixed schedules of reinforcement would be more motivating and rewarding because the rat can learn to predict the outcome of his work. Instead, Skinner found that the variable schedules were actually more motivating. The most telling result was that when the rewards ceased, the rats who were under the fixed schedule stopped working almost immediately, but those under the variable schedules kept working for a very long time.

If this reminds you of gambling, that's because gambling explicitly works under the very same schedule of variable reinforcement.

Go ahead, pull the "new email' lever. Take a chance. Most of the time you'll end up a loser, the proud recipient of yet another spam email, a press release you don't care about, or some irrelevant conversation someone has cc:ed you into. But not always. There are those rare few times when you'll hit the jackpot: you'll get an important bit of information you needed, or tentative contact from a long lost friend or associate, or other good news.

We're so ecstatic to get that single useful email out of hundreds that we can't keep ourselves from compulsively pressing the new email lever over and over and over, hoping it will happen again soon, like the caged rats in Skinners' experiments.

We desperately need to ask ourselves, and those around us, to revisit the purpose of email. Given what we know about the importance of flow to productive work, and how multi-tasking is largely a myth, is it worth the constant stream of minor interruptions?

We've overloaded email with so many meanings that it has imploded as a communication medium. Need an urgent answer to your question within a few minutes? Fire off a quick email and demand a response! Want to have a long back and forth discussion with several people? Email everyone! Do you have a new theory that you desperately want to explain to someone? Send it to them via email! Got a funny joke or picture you're dying to share? Email it to the office alias!

When we treat email as the kitchen sink of communication, appropriate for everything, it simply ceases to work at all.

Kathy Sierra was concerned that Twitter had the same variable reinforcement problem, but I think Twitter is in fact part of the answer to the problem.

Stop. Sending. Email.

Instead of abusing email as a "one size fits all" conduit for communication, be smart. Know when to escalate your communication to the right medium for the particular message you're trying to deliver:

Broad kudos? Post it on a feedback forum or your blog.
Need an urgent, immediate answer? Pick up the phone and call.
Got something that needs a lot of touchy feely discussion? Set up a face to face meeting.
Discussing a particular topic or product? Post it on a public message board.
Is this more of a friendly, social thing? Try using a social network like Twitter or Facebook.
Business proposal? Perhaps it would be smart to approach indirectly, through soliciting recommendations of business associates.

The real solution here is to move people beyond email silos wherever and whenever possible. Some amount of email is still inevitable, though. What steps can we take to turn our email from a dangerous variable reinforcement machine to something more … sane? Predictable, even?

Turn off all notification and interruption features in your email client.
Only check your email at regular, scheduled intervals.
Set up your email client to automatically highlight those emails from friends and business associates who are historically known to send you useful email.

Before you send that next email -- or press the "retrieve mail" button again -- ask yourself: do I smell a rat?

[advertisement] Interested in agile? See how a world-leading software vendor is practicing agile.

9 Ways Marketing Weasels Will Try to Manipulate You

Jeff Atwood — Thu, 10 Sep 2009 10:51:36 -0800

I recently read Predictably Irrational.

It's a fascinating examination of why human beings are wired and conditioned to react irrationally. We human beings are a selfish bunch, so it's all the more surprising to see how easily we can be manipulated to behave in ways that run counter to our own self-interest.

This isn't just a "gee-whiz" observation; understanding how and why we behave irrationally is important. If you don't understand how these irrational behaviors are triggered, the marketing weasels will use them against you.

In fact, it's already happening. Witness 10 Irrational Human Behaviors and How to Leverage Them to Improve Web Marketing. Don't say I didn't warn you.

Let's take a look at the various excerpts presented in that article, and consider how we can avoid falling into the rut of predictably irrational behavior -- and defend ourselves from those vicious marketing weasels.

1. Encourage false comparisons

When Williams-Sonoma introduced bread machines, sales were slow. When they added a "deluxe" version that was 50% more expensive, they started flying off the shelves; the first bread machine now appeared to be a bargain
When contemplating the purchase of a $25 pen, the majority of subjects would drive to another store 15 minutes away to save $7. When contemplating the purchase of a $455 suit, the majority of subjects would not drive to another store 15 minutes away to save $7. The amount saved and time involved are the same, but people make very different choices. Watch out for relative thinking; it comes naturally to all of us.

Realize that some premium options exist as decoys -- that is, they are there only to make the less expensive options look more appealing, because they're easy to compare. Don't make binding decisions solely based on how easy it is to compare two side-by-side options from the same vendor. Try comparing all the alternatives, even those from other vendors.
Don't be swayed by relative percentages for small dollar amounts. Yes, you saved 25%, but how much effort and time did you expend on that seven bucks?

2. Reinforce Anchoring

Savador Assael, the Pearl King, single-handedly created the market for black pearls, which were unknown in the industry before 1973. His first attempt to market the pearls was an utter failure; he didn't sell a single pearl. So he went to his friend, Harry Winston, and had Winston put them in the window of his 5th Avenue store with an outrageous price tag attached. Then he ran full page ads in glossy magazines with black pearls next to diamonds, rubies, and emeralds. Soon, black pearls were considered precious.
Simonsohn and Loewenstein found that people who move to a new city remain anchored to the prices they paid in their previous city. People who move from Lubbock to Pittsburgh squeeze their families into smaller houses to pay the same amount. People who move from LA to Pittsburgh don't save money, they just move into mansions.

Scale your purchases to your needs, not your circumstances or wallet size. What do you actually use? How much do you use it, and how frequently?
Try to objectively measure the value of what you're buying; don't be tricked into measuring relative to similar products or competitors. How much does buying this save you or your company? How much benefit will you get out of it? Attempt to measure that benefit by putting a concrete dollar amount on it.

3. It's "Free"!

Ariely, Shampanier, and Mazar conducted an experiment using Lindt truffles and Hershey's Kisses. When a truffle was $0.15 and a kiss was $0.01, 73% of subjects chose the truffle and 27% the Kiss. But when a truffle was $0.14 and a kiss was free, 69% chose the kiss and 31% the truffle.
According to standard economic theory, the price reduction shouldn't have lead to any behavior change, but it did.
Ariely's theory is that for normal transactions, we consider both upside and downside. But when something is free, we forget about the downside. "Free" makes us perceive what is being offered as immensely more valuable than it really is. Humans are loss-averse; when considering a normal purchase, loss-aversion comes into play. But when an item is free, there is no visible possibility of loss

You will tend to overestimate the value of items you get for free. Resist this by viewing free stuff skeptically rather than welcoming it with open arms. If it was really that great, why would it be free?
Free stuff often comes with well hidden and subtle strings attached. How will using a free service or obtaining a free item influence your future choices? What paid alternatives are you avoiding by choosing the free route, and why?
How much effort will the free option cost you? Are there non-free options which would cost less in time or effort? How much is your time worth?
When you use a free service or product, you are implicitly endorsing and encouraging the provider, effectively beating a path to their door. Is this something you are comfortable with?

4. Exploit social norms

The AARP asked lawyers to participate in a program where they would offer their services to needy employees for a discounted price of $30/hour. No dice. When the program manager instead asked if they'd offer their services for free, the lawyers overwhelmingly said they would participate

Companies may appeal to your innate sense of community or public good to convince you to do their work at zero pay. Consider carefully before choosing to participate; what do you get out of contributing your time and effort? Is this truly a worthy cause? Would this be worth doing if it was a paid gig?
When it comes to the web, make sure you aren't being turned into a digital sharecropper.

5. Design for Procrastination

Ariely conducted an experiment on his class. Students were required to write three papers. Ariely asked the first group to commit to dates by which they would turn in each paper. Late papers would be penalized 1% per day. There was no penalty for turning papers in early. The logical response is to commit to turning all three papers in on the last day of class. The second group was given no deadlines; all three papers were due in the last day of class. The third group was directed to turn their papers in on the 4th, 8th, and 12th weeks.
The results? Group 3 (imposed deadlines) got the best grades. Group 2 (no deadlines) got the worst grades, and Group 1 (self-selected deadlines) finished in the middle. Allowing students to pre-commit to deadlines improved performance. Students who spaced out their commitments did well; students who did the logical thing and gave no commitments did badly.

Steer clear of offers of low-rate trial periods which auto-convert into automatic recurring monthly billing. They know that most people will procrastinate and forget to cancel before the recurring billing kicks in.
Either favor fixed-rate, fixed-term plans -- or become meticulous about cancelling recurring services when you're not using them.

6. Utilize the Endowment Effect

Ariely and Carmon conducted an experiment on Duke students, who sleep out for weeks to get basketball tickets; even those who sleep out are still subjected to a lottery at the end. Some students get tickets, some don't. The students who didn't get tickets told Ariely that they'd be willing to pay up to $170 for tickets. The students who did get the tickets told Ariely that they wouldn't accept less than $2,400 for their tickets.
There are three fundamental quirks of human nature. We fall in love with what we already have. We focus on what we might lose, rather than what we might gain. We assume that other people will see the transaction from the same perspective as we do.

The value of what you've spent so far on a service, product, or relationship -- in effort or money -- is probably far less than you think. Be willing to walk away.
Once you've bought something, never rely on your internal judgment to assess its value, because you're too close to it now. Ask other people what they'd pay for this service, product, or relationship. Objectively research what others pay online.

7. Capitalize on our Aversion to Loss

Ariely and Shin conducted an experiment on MIT students. They devised a computer game which offered players three doors: Red, Blue, and Green. You started with 100 clicks. You clicked to enter a room. Once in a room, each click netted you between 1-10 cents. You could also switch rooms (at the cost of a click). The rooms were programmed to provide different levels of rewards (there was variation within each room's payoffs, but it was pretty easy to tell which one provided the best payout).
Players tended to try all three rooms, figure out which one had the highest payout, and then spend all their time there. (These are MIT students we're talking about). Then, however, Ariely introduced a new wrinkle: Any door left unvisited for 12 clicks would disappear forever. With each click, the unclicked doors shrank by 1/12th.
Now, players jumped from door to door, trying to keep their options open.They made 15% less money; in fact, by choosing any of the doors and sticking with it, they could have made more money.
Ariely increased the cost of opening a door to 3 cents; no change--players still seemed compelled to keeping their options open. Ariely told participants the exact monetary payoff of each door; no change. Ariely allowed participants as many practice runs as they wanted before the actual experiment; no change. Ariely changed the rules so that any door could be "reincarnated" with a single click; no change.
Players just couldn't tolerate the idea of the loss, and so they did whatever was necessary to prevent their doors from closing, even though disappearance had no real consequences and could be easily reversed. We feel compelled to preserve options, even at great expense, even when it doesn't make sense.

If your choices are artificially narrowed, don't passively get funneled towards the goal they're herding you toward. Demand choice, even if it means switching vendors or allegiances.
Don't pay extra for options, unless you can point to hard evidence that you need those options. Some options exist just to make you doubt yourself, so you'll worry about not having them.

8. Engender Unreasonable Expectations

Ariely, Lee, and Frederick conducted yet another experiment on MIT students. They let students taste two different beers, and then choose to get a free pint of one of the brews. Brew A was Budweiser. Brew B was Budweiser, plus 2 drops of balsamic vinegar per ounce.
When students were not told about the nature of the beers, they overwhelmingly chose the balsamic beer. When students were told about the true nature of the beers, they overwhelmingly chose the Budweiser. If you tell people up front that something might be distasteful, the odds are good they'll end up agreeing with you--because of their expectations.

Whatever you've heard about a brand, company, or product -- there's no substitute for your own hands-on experience. Let your own opinions guide you, not the opinions of others.
Just because something is labelled "premium" or "pro" or "award-winning" doesn't mean it is. Research these claims; don't let marketing set your expectations. Rely on evidence and facts.

9. Leverage Pricing Bias

Ariely, Waber, Shiv, and Carmon made up a fake painkiller, Veladone-Rx. An attractive woman in a business suit (with a faint Russian accent) told subjects that 92% of patients receiving VR reported significant pain relief in 10 minutes, with relief lasting up to 8 hours.
When told that the drug cost $2.50 per dose, nearly all of the subjects reported pain relief. When told that the drug cost $0.10 per dose, only half of the subjects reported pain relief. The more pain a person experienced, the more pronounced the effect. A similar study at U Iowa showed that students who paid list price for cold medications reported better medical outcomes than those who bought discount (but clinically identical) drugs.

Price often has nothing to do with value. Expensive is not synonymous with quality. Investigate whether the price is justified; never accept it at face value.
Don't fall prey to the "moneymoon"; just because you paid for something doesn't mean it's automatically worthwhile. Not everything we pay money for works well, or was even worth what we spent for it. We all make mistakes when buying things, but we don't want to admit it.

What I learned from Predictably Irrational is that everyone is irrational sometimes, and that's OK. We're not perfectly logical Vulcans, after all. The trick is training yourself to know when you're most likely to make irrational choices, and to resist those impulses.

If you aren't at least aware of our sad, irrational human condition, well ... that's exactly where the marketing weasels want you.

[advertisement] Interested in agile? See how a world-leading software vendor is practicing agile.

If It Looks Corporate, Change It

Jeff Atwood — Wed, 02 Sep 2009 23:59:59 -0800

Are you familar with happy talk?

If you're not sure whether something is happy talk, there's one sure-fire test: if you listen very closely while you're reading it, you can actually hear a tiny voice in the back of your head saying "Blah blah blah blah blah...."
A lot of happy talk is the kind of self-congratulatory promotional writing that you find in badly written brochures. Unlike good promotional copy, it conveys no useful information, and focuses on saying how great we are, as opposed to delineating what makes us great.

Happy talk is the kudzu of the internet; the place is lousy with the stuff.

And then there's the visual equivalent of happy talk. Those cloying, meaningless stock photos of happy users doing ... something ... with a computer.

What is going on here? Given the beatific expressions, you'd think they were undergoing some kind of nerd rapture. Maybe they're getting a sneak preview of the singularity, I don't know.

It's unclear to me why companies (and even some individuals) think they need happy talk, stock photos of multicultural computer users, or the occasional headset hottie. Jason Cohen provides an explanation:

Even before I had a single customer, I "knew" it was important to look professional. My website would need to look and feel like a "real company." I need culture-neutral language complimenting culturally-diverse clip-art photos of frighteningly chipper co-workers huddled around a laptop, awash with the thrill and delight of configuring a JDBC connection to SQL Server 2008.
It also means adopting typical "marketing-speak," so my "About Us" page started with:

Smart Bear is the leading provider of enterprise version control data-mining tools. Companies world-wide use Smart Bear's Code Historian software for risk-analysis, root-cause discovery, and software development decision-support.

"Leading provider?" "Data mining?" I'm not even sure what that means. But you have to give me credit for an impressive quantity of hyphens.
That's what you're supposed to do, right? That's what other companies do, so it must be right. Who am I to break with tradition?

I'm not sure where we got our ideas about this stuff, but it is true that some large companies promote a kind of doublespeak "professionalism". Kathy Sierra describes her experiences at Sun:

By the time I got to Sun, using the word "cool" in a customer training document was enough to warrant an entry in your annual performance eval. And not in a good way.
I cannot count the times I heard the word "professionalism" used as justification for why we couldn't do something. But I can count the few times I heard the word "passion" used in a meeting where the goal was to get developers to adopt our newest Java technologies. What changed?
Some argue that by maintaining strict professionalism, we can get the more conservative, professional clients and thus grow the business. Is this true? Do we really need these clients? Isn't it possible that we might even grow more if we became braver?

It's a shame that this misguided sense of professionalism is sometimes used as an excuse to put up weird, Orwellian communication barriers between yourself and the world. At best it is a facade to hide behind; at worst it encourages us to emulate so much of what is wrong with large companies. Allow me to paraphrase the simple advice of Elmore Leonard:

If it looks corporate, change it.

The next time you find yourself using professional text, or professional stock images, consider the value of this "professionalism". Is it legitimately helping you communicate? Or is it getting in the way?

Have You Met Your Dog, Patches?

Jeff Atwood — Thu, 27 Aug 2009 23:59:59 -0800

The Gamasutra article Dirty Coding Tricks is a fantastic read. One part of it in particular rang true for me.

Consider the load of pain I found myself in when working on a conversion of a 3D third person shooter from the PC to the original PlayStation.
Now, the PS1 has no support for floating point numbers, so we were doing the conversion by basically recompiling the PC code and overloading all floats with fixed point. That actually worked fairly well, but were it fell apart was in collision detection.
The level geometry that was supplied to us worked reasonably well in the PC version of the game, but when converted to fixed point, all kinds of seams, T-Junctions and other problems were nudged into existence by the microscopic differences in values between fixed and floats. This problem would manifest itself by the main character (called "Damp") simply falling through those tiny holes, into the abyss below the level.
We patched the holes we found, tweaking the geometry until Damp no longer fell through. But then the game went into test at the publisher, and suddenly a flood of "falling off the world" bugs were delivered to us. Every day a fresh batch of locations were found with these little holes that Damp could slip through. We would fix that bit of geometry, then the next day they would send us ten more. This went on for several days. The publisher's test department employed one guy whose only job was to jump around the world for ten hours a day, looking for places he could fall through.
The problem here was that the geometry was bad. It was not tight and seamless geometry. It worked on the PC, but not on the PS1, where the fixed point math vastly magnified the problems. The ideal solution was to fix the geometry to make it seamless.
However, this was a vast task, impossible to do in the time available with our limited resources, so we were relying on the test department to find the problem areas for us.

There's never time to do it right, but there's always time to do it over. If this sounds familiar, you're not alone. I have at times found myself slipping into this pattern, continually patching the same code over and over. It's the kind of code that, when submitted for code review, you're tempted to self-deprecatingly introduce as have you met my dog, patches?

While "patchiness" isn't always a bad thing -- the venerable Apache HTTP Server is testament to that -- it's probably an exception.

the original FAQ on the Apache Server project's website, from 1996 to 2001, claimed that "The result after combining [the NCSA httpd patches] was a patchy server."

Reading the Gamasutra article shamed me into to attacking a section of extra-patchy Stack Overflow code. Code which I constantly had to tweak in various ongoing ways to get it to work right. But first, I had to get over my fear. Fear. That's what led to all the patching in the first place. It was obvious this was working code which had become crufty over time, but it was working. And it's one of the core bits of functionality in the site. In those circumstances, it's easy to mentally justify "just this small change, just this once" rather than the fundamental rewrite you really need.

When you finally realize you've spent the last six months telling yourself the same "just this small change, just this once" lie, then you've met your dog patches, too.

Now what are you going to do about that rascally scamp?

[advertisement] Interested in agile? See how a world-leading software vendor is practicing agile.

That Means It's Working

Jeff Atwood — Mon, 24 Aug 2009 23:59:59 -0800

We may kid ourselves into thinking we're writing out of some sense of public good, or to create connections, or contribute some small bit of knowledge to the world. But let's face it. Most of us blog because we're raving egomaniacs. We not only love to hear ourselves talk, we're incredibly eager to hear other people talk about us, and the more the better. I think Dale Carnegie put it best.

Nothing is sweeter to someone's ears than their own name.

So it should come as no surprise that I have an automatic Google ego search set up for my name. Nothing special about that. It is considered neighborly to have your ear to the ground (within reason), and to politely comment on relevant articles mentioning you and your "stuff". All very standard, banal, ego-fluffing stuff.

But of all the mentions I've gotten, nobody has utterly nailed it in the way that Brian Gianforcaro has.

Right on. That's one thing you and I have in common, burny.

As a software developer, you are your own worst enemy. The sooner you realize that, the better off you'll be. In fact, that's the tipping point between amateurs and professionals in our industry: the professionals realize everything they write sucks.

So, to the extent that I can become a conduit for other programmers to have that same epiphany in their own programming careers, that means it's working.

[advertisement] Interested in agile? See how a world-leading software vendor is practicing agile.

The Only Truly Failed Project

Jeff Atwood — Wed, 19 Aug 2009 23:59:59 -0800

Do you remember Microsoft Bob? If you do, you probably remember it as an intensely marketed but laughable failure -- what some call the "number one flop" at Microsoft.

There's no question that Microsoft Bob was nothing short of an unmitigated disaster. But that's the funny thing about failures -- they often lead to later successes. Take it from someone who lived and breathed the Bob project:

I was the one who sent Bill Gates email at the height of the positive Bob-mania that said we were likely to face a horrible backlash. Tech influentials had started telling me that they were going to bury Bob. They not only didn't like it, they were somehow angry that it had even been developed. It was personal.
And that's exactly what happened. Bob got killed. But first, it was ridiculed and stomped.
For Microsoft, it was a costly mistake. For the people who worked on it, Bob taught many lessons. Lessons that came into play for subsequent products that made a big impact, both at Microsoft and beyond.
How many people know that the lead developer for Bob 2.0 was also the co-founder of Valve and the development lead for Half-Life, which became an industry phenomenon, winning more than 50 Game of the Year awards and selling more than 10 million copies?
Or that Darrin Massena - development lead for Bob 1.0, most recently named Technical Innovator of the Year here in Washington State - and Valve co-founder Mike Harrington are the co-founders and partners behind Picnik - which is now the world's leading online photo editor, attracting almost 40 million visits a month and a million unique users a day.

And then, of course, I'd be remiss if I didn't mention that Melinda French -- Bill Gates' future wife -- managed the Microsoft Bob project at one point. Bob was the first Microsoft consumer project that Bill Gates personally had a hand in launching. Well, at least he got a wife out of it.

Yes, Bob was an obvious, undisputed and epic failure. We can point and laugh at Bob. But to me, Bob is less of a comic figure than a tragic one.

Unless you're an exceptionally lucky software developer, you've probably worked on more projects that failed than projects that succeeded. Failure is de rigeur in our industry. Odds are, you're working on a project that will fail right now. Oh sure, it may not seem like a failure yet. Maybe it'll fail in some completely unanticipated way. Heck, maybe your project will buck the odds and even succeed.

But I doubt it.

I own a boxed copy of Microsoft Bob. I keep it on my shelf to remind me that these kinds of relentless, inevitable failures aren't the crushing setbacks they often appear from the outside. On the contrary; I believe it's impossible to succeed without failing.

Charles Bosk, a sociologist at the University of Pennsylvania, once conducted a set of interviews with young doctors who had either resigned or been fired from neurosurgery-training programs, in an effort to figure out what separated the unsuccessful surgeons from their successful counterparts.
He concluded that, far more than technical skills or intelligence, what was necessary for success was the sort of attitude that Quest has -- a practical-minded obsession with the possibility and the consequences of failure. "When I interviewed the surgeons who were fired, I used to leave the interview shaking," Bosk said. "I would hear these horrible stories about what they did wrong, but the thing was that they didn't know that what they did was wrong. In my interviewing, I began to develop what I thought was an indicator of whether someone was going to be a good surgeon or not. It was a couple of simple questions: Have you ever made a mistake? And, if so, what was your worst mistake? The people who said, 'Gee, I haven't really had one,' or, 'I've had a couple of bad outcomes but they were due to things outside my control' -- invariably those were the worst candidates. And the residents who said, 'I make mistakes all the time. There was this horrible thing that happened just yesterday and here's what it was.' They were the best. They had the ability to rethink everything that they'd done and imagine how they might have done it differently."

I recently watched the documentary Tilt: The Battle to Save Pinball.

It's a gripping story of a pinball industry in crisis. In order to save it, the engineers at Williams -- the only remaining manufacturer of pinball machines in the United States -- were given a herculean task: invent a new form of pinball so compelling that it makes all previous pinball machines seem obsolete. I don't want to spoil the whole documentary, so I'll gloss over exactly how that happened, but astoundingly enough -- they succeeded.

And then were promptly laid off en masse, as Williams shut down its pinball operations.

Unlike Microsoft Bob, the Williams engineers built an almost revolutionary product that was both critically acclaimed and sold well -- but none of that mattered. It's sobering to watch the end reel of Tilt, as the engineers involved mournfully discuss the termination of their bold and seemingly successful project.

Everyone was in awe. They couldn't understand why it happened. Here we'd just done this thing that from all we could tell was a total success. Why would they do that?
We succeeded. Management gave us an impossible goal, and we sat there and we actually did what they thought we couldn't do.
You know, we didn't really win... we lost. I gave it everything I had. I think that those fifty guys that worked on it, they also passionately did everything that they could.

Sometimes, even when your project succeeds, you've failed. Due to forces entirely beyond your control. It's depressing, but it's reality.

The trailout isn't all doom and gloom. It also documents the ways in which these talented pinball engineers went on to practice their craft after being laid off. Most of them still work in the video game or pinball industry. Some freelance. Others formed their own companies. A few went on to work at Stern Pinball, which figured out how to make a small number of pinball machines and still turn a profit.

These two stories, these two projects -- the abject failure of Microsoft Bob, and the aborted success of Pinball 2000 -- have something in common beyond mere failure. All the engineers involved not only survived these failures, but often went on to greater success afterwards. Possibly as a direct result of their work on these "failures".

Failure is a wonderful teacher. But there's no need to seek out failure. It will find you. Whatever project you're working on, consider it an opportunity to learn and practice your craft. It's worth doing because, well, it's worth doing. The journey of the project should be its own reward, regardless of whatever happens to lie at the end of that journey.

The only truly failed project is the one where you didn't learn anything along the way.

[advertisement] Interested in agile? See how a world-leading software vendor is practicing agile.

All Programming is Web Programming

Jeff Atwood — Fri, 14 Aug 2009 03:57:24 -0800

Michael Braude decries the popularity of web programming:

The reason most people want to program for the web is that they're not smart enough to do anything else. They don't understand compilers, concurrency, 3D or class inheritance. They haven't got a clue why I'd use an interface or an abstract class. They don't understand: virtual methods, pointers, references, garbage collection, finalizers, pass-by-reference vs. pass-by-value, virtual C++ destructors, or the differences between C# structs and classes. They also know nothing about process. Waterfall? Spiral? Agile? Forget it. They've never seen a requirements document, they've never written a design document, they've never drawn a UML diagram, and they haven't even heard of a sequence diagram.
But they do know a few things: they know how to throw an ASP.NET webpage together, send some (poorly done) SQL down into a database, fill a dataset, and render a grid control. This much they've figured out. And the chances are good it didn't take them long to figure it out.
So forgive me for being smarmy and offensive, but I have no interest in being a 'web guy'. And there are two reasons for this. First, it's not a challenging medium for me. And second, because the vast majority of Internet companies are filled with bad engineers - precisely because you don't need to know complicated things to be a web developer. As far as I'm concerned, the Internet is responsible for a collective dumbing down of our intelligence. You just don't have to be that smart to throw up a webpage.
I really hope everybody's wrong and everything doesn't "move to the web." Because if it does, one day I will either have to reluctantly join this boring movement, or I'll have to find another profession.

Let's put aside, for the moment, the absurd argument that web development is not challenging, and that it attracts sub-par software developers. Even if that was true, it's irrelevant.

I hate to have to be the one to break the bad news to Michael, but for an increasingly large percentage of users, the desktop application is already dead. Most desktop applications typical users need have been replaced by web applications for years now. And more are replaced every day, as web browsers evolve to become more robust, more capable, more powerful.

You hope everything doesn't "move to the web"? Wake the hell up! It's already happened!

Any student of computing history will tell you that the dominance of web applications is exactly what the principle of least power predicts:

Computer Science spent the last forty years making languages which were as powerful as possible. Nowadays we have to appreciate the reasons for picking not the most powerful solution but the least powerful. The less powerful the language, the more you can do with the data stored in that language. If you write it in a simple declarative from, anyone can write a program to analyze it. If, for example, a web page with weather data has RDF describing that data, a user can retrieve it as a table, perhaps average it, plot it, deduce things from it in combination with other information. At the other end of the scale is the weather information portrayed by the cunning Java applet. While this might allow a very cool user interface, it cannot be analyzed at all. The search engine finding the page will have no idea of what the data is or what it is about. The only way to find out what a Java applet means is to set it running in front of a person.

The web is the very embodiment of doing the ~~stupidest~~simplest thing that could possibly work. If that scares you -- if that's disturbing to you -- then I humbly submit that you have no business being a programmer.

Should all applications be web applications? Of course not. There will continue to be important exceptions and classes of software that have nothing to do with the web. But these are minority and specialty applications. Important niches, to be sure, but niches nonetheless.

If you want your software to be experienced by as many users as possible, there is absolutely no better route than a web app. The web is the most efficient, most pervasive, most immediate distribution network for software ever created. Any user with an internet connection and a browser, anywhere in the world, is two clicks away from interacting with the software you wrote. The audience and reach of even the crappiest web application is astonishing, and getting larger every day. That's why I coined Atwood's Law.

Atwood's Law: any application that can be written in JavaScript, will eventually be written in JavaScript.

Writing Photoshop, Word, or Excel in JavaScript makes zero engineering sense, but it's inevitable. It will happen. In fact, it's already happening. Just look around you.

As a software developer, I am happiest writing software that gets used. What's the point of all this craftsmanship if your software ends up locked away in a binary executable, which has to be purchased and licensed and shipped and downloaded and installed and maintained and upgraded? With all those old, traditional barriers between programmers and users, it's a wonder the software industry managed to exist at all. But in the brave new world of web applications, those limitations fall away. There are no boundaries. Software can be everywhere.

Web programming is far from perfect. It's downright kludgy. It's true that any J. Random Coder can plop out a terrible web application, and 99% of web applications are absolute crap. But this also means the truly brilliant programmers are now getting their code in front of hundreds, thousands, maybe even millions of users that they would have had absolutely no hope of reaching pre-web. There's nothing sadder, for my money, than code that dies unknown and unloved. Recasting software into web applications empowers programmers to get their software in front of someone, somewhere. Even if it sucks.

If the audience and craftsmanship argument isn't enough to convince you, consider the business angle.

You're doing a web app, right? This isn't the 1980s. Your crummy, half-assed web app will still be more successful than your competitor's most polished software application.

Pretty soon, all programming will be web programming. If you don't think that's a cause for celebration for the average working programmer, then maybe you should find another profession.

[advertisement] Interested in agile? See how a world-leading software vendor is practicing agile.