al3x

So You're Moving to San Francisco

2009-10-04T00:00:00-07:00

So You’re Moving to San Francisco

Writing about a place is difficult. You can spend months, years, even a lifetime in a city and still not really know it. More challenging still, everyone experiences a place differently. Two people who’ve grown up in the same place might fundamentally disagree on what the most scenic landmarks are, if the locals are friendly, the best places to eat, and so on.

I’ve been in San Francisco for over a couple of years now. I’d hardly say I know the place exhaustively, but I know it well enough to have a moderately informed opinion. The purpose of this post is to share my particular opinion about the city so that a like-minded individual who is considering living here might have an additional perspective. I’m assuming that my audience for this post is largely other twenty-somethings considering moving to SF to work in the tech industry. I’m assuming that you’ve read at least a couple other posts by me, and (astoundingly) don’t think I’m a total dolt.

This is not intended as a persuasive piece, and is particularly not intended for native San Franciscans. If you enjoy living in San Francisco, stop reading right now. I’m dead serious. If you don’t stop reading, you’ll probably come across something you disagree with, then you’ll want to leave a nasty comment, then you’ll realize I don’t accept comments, then you’ll email me, and then I’ll have to ignore your email because I warned you not to read this. This post really isn’t for you. If you like San Francisco, go write about the reasons why on your own site. Seriously. Stop reading.

So. You’ve made it through the caveats. Let’s get into it.

First, The Conclusion

I’m going to skip right to the heart of what I want to say about this city: if you’ve never lived in a major city before, you’ll probably like San Francisco. However, if you’re coming from another notable city, you may be disappointed. Hopefully, that’s pretty uncontroversial.

If you’re moving from, say, a New York or a Chicago or a London, you may end up loving San Francisco for its climate, its diversity, its food, or above all, its unique mindset. That mindset – decidedly Left Coast, laid back but not lazy, accepting of oddities and oddballs, always appreciating diamonds in the rough – resonates perfectly with some special people who are destined to be lifelong San Franciscans. If that way of thinking doesn’t jive with you, chances are good that, like me, you’ll just be passing through. Without appreciating that mindset, I don’t think the pros of the city outweigh the cons.

With all that out front, let’s talk about the niceties and the frustrations of the city.

The Good

If you’ve ever followed my tweets, you probably know that I live for good food and drink. When I came to San Francisco, my standards had been set reasonably high by the always-improving food scene in my hometown of Washington, DC, as well as by travels to other foodie destinations.

I can say without reservation that San Francisco is a great city for food. Everything from hole-in-the-wall ethnic dives to Michelin rated dining is well represented. The proximity to Napa means there’s always good wine and a strong French influence to compliment the uniquely Californian approach to cuisine. It’s entirely possible to have a bad meal in San Francisco, but to do so is entirely your fault: Yelp and other social recommendation services are better represented here than anywhere else in the world, and a suggestion for a good restaurant is always just a couple clicks (or taps) away.

SF is also a great town for coffee and cocktails, two of my other vices. You may not find a good coffee shop or bar in every neighborhood, but most of the trendier neighborhoods have at least one or the other. Similarly, you’ll find good wine bars around the city. Beer is somewhat underrepresented for such a major city (as I’ve written about previously), but you won’t go without. Cocktails are really the city’s standout for me, though. SF’s bartenders are not playing around.

San Francisco boasts superb weather. Particularly if you come from the East Coast or Midwest, you’ll find that you can basically wear the same thing all year ‘round. A uniform of jeans, a t-shirt, and a hoodie or light jacket will basically get you through the entire thirty degree range of temperatures that occur in the city’s various microclimates. There may be some fog, or a bit of rain from time to time, but most of the time it’s sunny and hovering around the high 60s to lower 70s. This means you can basically always bike to where you want to go without any special gear, if that’s your thing.

You won’t want for music, either. Again, SF doesn’t compete with other major cities in this regard, but there’s a healthy variety of musical styles represented and a fair diversity of venues to choose from. Music festivals are frequent, and there are plenty of opportunities to join a band or DJ at a club night if you’ve got the talent and time.

If you’re coming here to make your startup dreams come true or land that Google job you’ll find that the sheer concentration of tech industry professionals makes the Bay Area a kind of mecca for driven geeks. You can spend every night of the week at an industry party or programmer meet-up. You’ll meet the people who write the blogs you love and develop the software and hardware you use and admire. You’ll generally feel like you’re with your kind. And if the city itself doesn’t have enough tech for you, Silicon Valley is less than an hour away. For me, the tech community is going to be the hardest thing about San Francisco to leave behind.

Food, drink, weather, music, and maybe the tech industry, if it’s applicable to you. That’s the good stuff. Now, take a deep breath and let’s explore the city’s darker side.

The Bad

Just as the density of high tech in San Francisco is a boon, it can also a burden for geeks. As my social circle grew after moving to the city, I began to feel as if I always had to be “on” – always representing my job, always receptive to talking tech and hearing a stranger’s latest pitch. It’s easy to meet people through the tech scene in SF, but they’re professional contacts, not friends. I now count some of those contacts as true friends, but real friendships take work and time. When you first move here, it’s easy to confuse knowing people through your job or technical interests with having a solid social network. Just remember that someone who added you on LinkedIn isn’t going to help you out when you’re sick, or moving, or just need someone to talk to frankly without worrying about leaking a trade secret.

San Francisco is also, perhaps infamously, an intellectual and cultural bubble in which ludicrous ideas can find support, particularly in the tech industry. Before long, you may find yourself nodding in sincere agreement as someone explains the inane first-world problem that their startup or pet open source project is trying to solve. It’s hard work to maintain perspective and not get caught up in a way of thinking that privileges the desires of young white men with high technical proficiency and lots of disposable income. But then, this is a double-edge sword: some ideas that seem silly at the outset have world-changing, democratizing potential (I’d like to think Twitter is one such idea, of course). Be open, but skeptical.

There are far more fundamental problems with the city than the tech industry bubble. Perhaps the most visceral is that, for a first world city, San Francisco is dirty. No, filthy. No, disgusting. Whenever I travel outside of San Francisco, I’m amazed at what a disastrous anomaly it is. Sidewalks are routinely covered in broken glass, trash, old food, and human excrement. The smell of urine is not uncommon, nor is the sight of homeless persons in varying states of dishevelment. I frequented tough neighborhoods in DC and Baltimore – then the murder capital of the nation – and only in San Francisco have I been actively threatened on the street.

What sickens me most about San Francisco is not its dirt, or its large homeless population, or its questionable safety, but that locals and the city government seem to accept these circumstances. Hipsters boast of how disgusting and unsafe their Mission living situations are, as if choosing to live amongst squalor when you have the means not to do so makes you a better person. The wealthy seclude themselves in the Marina, Russian Hill, and Pacific Heights, and lobby against public transportation that would bring undesirables to their pristine neighborhoods. Aging hippies in the Haight argue about marijuana legalization and anti-war referendums when men and women are dying – visibly dying – on the streets of the Tenderloin. It’s as if all parties don’t occupy the same city, see the same shameful sights on the street, and bear the same responsibilities to taxes and charity that might help address these deep-seated and difficult problems.

Month after month, San Franciscans gather for festivals and parades: Pride, the Folsom Street Faire, LoveFest, Bay to Breakers, and so forth. The privileged fill the streets, dressed in gaudy costumes, embracing any excuse to celebrate their sexuality, their liberal views, their comfort with alternative approaches to life and social structures. Were San Francisco taking care of its own, creating an environment in which everyone had access to the same comforts and opportunities, I would encourage such celebrations every week. But, as liberal and libertarian as I am, I think there’s something disturbing and solipsistic and fundamentally broken about a place that seems to value a different way of life over better quality of life. It is this that I object to most strenuously about San Francisco.

There are other nuisances and disappointments, to be sure:

An obscenely high cost of living for comparatively poor real estate and social services.
Unreliable and inadequate public transit, paling in comparison to most any other major city in the world.
Lots of traffic and very little parking – factors that would be less of an issue if the public transit was adequate.
Generally poor urban/civic planning.
Limited and mediocre cultural institutions. It’s easy to exhaust museums, theater, and other forms of the arts in SF. Most of what you’ll find outside the mainstream is dim, amateurish, and – as above – obsessed with being different rather than simply being better. (The ballet is the major exception. It’s quite good.)
Entirely a matter of personal preference, but I dislike much of the architecture in San Francisco. Some find the endless peeling Victorians quaint. I prefer buildings that are truly historic or aggressively modern.
Vast dead spaces between and within neighborhoods. For a city of relatively small size, you’ll find that most of it isn’t worth repeated visits. Areas worth spending time in are usually just several blocks, scarcely enough to occupy an hour or two with window shopping and a stroll.
Enormous competition for limited resources. You will wait for everything. The better a thing is (food, coffee, a nice place to sit), the longer you’ll wait for it. When you finally get what you want, you’ll be crammed in with others trying to enjoy the same place/thing, diminishing everyone’s enjoyment.

There is, I’ve found, precious little to do here, particularly if you’re not inclined towards sports or the outdoors. I recall asking several locals what exactly people did on a Saturday afternoon, at a loss after having gone to the scant few museums and walked around the few worthwhile neighborhoods. “Hang out in the park or sleep, I guess” was the common answer. And indeed, that’s what many people do: the Mission’s Dolores Park is filled with idling throngs weekend after weekend, soaking up the sun, chatting, drinking, smoking, existing. Nothing wrong with the simple pleasures of friends and good weather, but there’s more to life than living from one hangout to the next.

There are some things about the city that are harder to put a finger on, too. While people in San Francisco are endearingly open-minded, all too often they’re self-centered, passive aggressive, and cold. As above, it’s easy to meet people through work or a common interest, but harder to meet random friendly strangers. Rarely in San Francisco has a kindness been done to me by a stranger – offering directions when I look lost, for example. When traveling, I’m again shocked at how much better people are to one another in other places, even in reputedly hard and unfriendly cities like New York.

Finally

One has to ask, after all that: why are you still here? The answer is that I’m in San Francisco for as long as my work requires me to be. Once I’m able to work remotely with confidence, either for Twitter or another employer, I have every intention of moving with my fiancée and two cats to Portland, Oregon, a place which I feel/hope better reflects my values. Quite simply, I want to live somewhere that works, and San Francisco feels broken. Portland doesn’t work perfectly, particularly in terms of its high unemployment, but it feels closer to what I want in a place than any other city I’ve visited. I’ll miss San Francisco’s strong tech community and other things about the city, but I can’t say I anticipate much reminiscence about the place.

I’m aware that all this is weighted more heavily towards the negative, but I don’t know that I should apologize for having had a negative experience of the city. I may be a critic by nature, but I try to look for the good in people and places. I feel like I’ve given San Francisco a fair go of it, but found in the end that it’s just not for me. Hopefully, the above assessment is useful to a like-minded person trying to make a decision about the city.

If you do decide to move here, read Mat Honan’s guide to the ideal SF experience. Mat has been here way longer than I have, knows the city better, and loves it. I agree with his assessment of where and how to live here, although I ended up moving to a less authentically San Franciscan neighborhood (SOMA) after having tried a more traditional neighborhood. It’s all a matter of personal taste and finding what resonates with you.

Good luck.

Addenda

Monday, October 5th — I’ve gotten a surprising amount of feedback about this post, most of it positive. One email in particular, though, had a portion worth reprinting:

“For two years, I was the Development Director at a small SF health clinic that serves some of most under-privileged folks in the city. We participated in Folsom Street Fair’s beneficiary program – the gist of it is, money collected at the gates, and from the beverage booths, is divided amongst community based groups that apply to get a piece of the pie. It can be as little as $5000; sometimes, up to $20,000 per group. Folsom alone gives away a few hundred thousand dollars a year in this way. For struggling community orgs, especially those that rely on dwindling city and state funds, this money is crucial. Pride, LoveFest, and Castro Street Fair do the same.”

I’m glad to hear that the events I called out are giving back to their community; indeed, I assumed they probably were, as most such events tend to have a charitable arm in this age of consumer guilt offsets. That’s something.

The author of the above email mentioned that she has since moved to New York, frustrated by San Francisco’s seeming inability to get things done and improve itself.

Taking On a New Challenge At Twitter

2009-10-02T00:00:00-07:00

Taking On a New Challenge At Twitter

I started working on Twitter in January of 2007, just a few months after the site launched. I honestly didn’t think I’d still be working on it almost three years later. I moved to San Francisco in May of that year to work for Obvious Corp, which was intended as an incubator for web product ideas: work on a bunch of stuff, see what sticks, run with it. It just so happened to be that Obvious’ first endeavor stuck in a big way.

My first several months at Twitter were spent all over the map: tweaking user-facing features one day, working on our first attempts at caching the next. Taking a breather from code one afternoon, I decided to document and flesh out our nascent API. Back then, we had little more than a page that said, more or less, “poke at some URLs and see what works”.

Amazingly, creative developers had already started building things with nothing more than that nudge in the right direction. Still, I thought it couldn’t hurt to give the fledgling Twitter API a little love. I started actively participating in a Google Group for the API that someone had set up and forgotten about. Pretty soon, that discussion group became a thriving community where developers helped us shape a suite of APIs to meet the needs of all sorts of different applications.

At that time, I lived, ate, breathed, and (rarely) slept the Twitter API. Developers were pleasantly surprised to get a near-instant response to questions at 2AM on a Sunday night. We had the agility and freedom to take new API methods and features live within mere hours of a developer saying, “this would help me out”. Sometimes, this meant launching things that weren’t fully baked and having to pull them back in. Still, it was an exciting time. I was able to help something special grow and evolve.

Today

Fast-forward to today. Developers have built tens of thousands of applications that talk to what has now become the Twitter Platform, a suite of APIs that encompasses everything users can do on the Twitter site and much, much more. The diversity of the Twitter ecosystem is fantastic. Developers have built Twitter apps on every mobile, desktop, and web platform and written client libraries in just about any programming language you can name. They’ve leveraged our Search and Streaming APIs for data mining and research. People have helped to coordinate disaster relief efforts and supported charitable causes with the Twitter API. And, increasingly, the great ideas that third-party Twitter developers come up with are turning into well-funded and successful commercial businesses.

The team behind the Twitter Platform is also in a great place. Ryan Sarver, product manager for the Platform, is doing an incredible job working through our ambitious roadmap, exploring partnerships, and more. Talented engineers like Marcel Molina and Raffi Krikorian are cranking out fixes and new methods left and right, and we have more engineers joining the team soon. The team is committed to building a world class third-party developer support organization, and is keeping abreast of upcoming standards and how they fit into our technologies.

Working on the Twitter Platform has been incredibly fulfilling, and the APIs and the community around them are only going to get better. Now, while staying on at Twitter, it’s time for me to work on something new.

What’s Next

Starting in a week or so, I’m going to be working on new layers of Twitter’s infrastructure alongside the brilliant folks on our Application Services and Infrastructure teams. It’s an opportunity to get back into nitty-gritty programming, to learn from top-notch engineers, and to put more of what I learned while writing Programming Scala into practice.

I’m excited to take on a new challenge, to be working with great people, and to be at a company that gives its employees the freedom to pursue what’s interesting to them. If that sounds like a place you’d like to be, consider applying. We have lots of exciting work ahead of us at Twitter.

My Get-Back-To-Work Hack

2009-09-14T00:00:00-07:00

My Get-Back-To-Work Hack

I’ve been looking for a way to fight distractions on the web. I’ve tried SelfControl, but just getting a “server not found” error in the browser when I visit a temptingly unproductive site isn’t particularly motivational.

The hack for this I came up with today is cheap and dumb, but it works. If you want to be sure it’s worth your time, watch this 46 second long screencast, then read on if you like what you see.

If you’re an experienced web developer, this will probably take you all of ten minutes to set up. If not, I’ve tried to explain it in pretty friendly terminology, but you might want to grab the nearest techie to help you out.

Step One: Block Distracting Domains

The /etc/hosts file tells your computer where other computers are. Normally, this file is almost empty; it just points your computer at itself, reminding it that its local address is 127.0.0.1. Your computer doesn’t need to know much more than that because it uses DNS and similar protocols to find other computers over the network. Still, every time your computer tries to talk to another computer, it checks /etc/hosts to see if we’ve specified the other computer’s address.

We can make use of the /etc/hosts file to block specific domains that we shouldn’t be spending time on. We just associate those domains with a bad address – in this case, the address of our local machine.

My /etc/hosts looks like this. It’s the lines below the commented-out line about “distractions” that matter. Those lines say that the specified domains are served by my own computer. They aren’t, of course, so when I visit those sites, my browser gives me an error page.

Go ahead and put your own time-wasting sites in there; be sure to add the “www.” variant, if the site has one. If you like, you can collapse all the domains onto one line after the “127.0.0.1”, but I find it easier to read over multiple lines.

To an extent, this solves the problem: if you visit one of the domains you added to the file, you don’t get there. But as with SelfControl’s solution, an error page in your browser isn’t very motivational. Let’s go further.

Step Two: Serving Content for Blocked Domains

Since we’re directing requests for distracting domains to our own system, why not do something with those requests? Fire up a web server on the usual port (80). On a Mac, this is as simple as going to Sharing in System Preferences and ticking the box for Web Sharing. The text there may say that your computer’s website is available at some fancy address, but it’s also available on http://127.0.0.1/.

Now, visit one of those sites you put in your /etc/hosts. You get a real webpage this time, right? Most likely, it’s whatever default page came with your web server software; it might say “It worked!” or something equally inane. The name of this file is probably index.html.en (or maybe without the “.en” part). On a Mac, it lives at /Library/WebServer/Documents/index.html.en. You should modify that file. Make it pretty, but not too pretty. Put a message in there that’s motivational to you. Visit one of your blocked sites, and you should see your custom message instead. Pretty good.

There’s a problem, though. Say you blocked dumbsite.com. If you visit http://dumbsite.com/foo/bar/baz.html, you’ll get a 404 File Not Found error from your local web server. That’s because you don’t have the files and directories the server is looking for inside the default folder your web server stores web pages in. You don’t want to have to provide these, either; you just want to serve the same page for every domain you’ve blocked.

If you’re using Apache, you can use a couple of lines like this inside your httpd.conf to rewrite any URL to point to a file in your document root (in this case, index.php, but change that to whatever you want). That will get you back to seeing your custom page when you visit a blocked domain, regardless of the URL.

If you want to keep it simple, get all that working and stop here. You’ve got a nice custom reminder to get back to work, and few moving parts to maintain. If you’re comfortable setting up PHP, though, venture on for a little gravy.

Step Three: Doing Something Useful with Blocked Domains

I didn’t just want a motivational message. I wanted a reminder of exactly what I should be doing.

Once I got the above working, I flipped on PHP in my httpd.conf and set Apache to run as my user. Then, I changed my index.php to this. The key lines are at the bottom. They tell PHP to fire off a command, just as if I issued it from a terminal. In my setup, the command will fire up Things if it isn’t running, or move it to the front of my windows if it is.

Variations

There’s a bunch of ways you could do this differently. Rather than modifying /etc/hosts, you could firewall off any distracting domains. You could run your own HTTP proxy and do the filtering there. Or, rather than popping up an application, you could have that PHP script (or however you decide to implement it) pull your to-do list right into the admonishing web page. Or, you could, y’know, just use plain old real actual self control. But that’s easier said than done, I’ve found.

What you do with the block page is really up to you. All I know is, popping up my to-do list works for me as motivation to get something done. I took a couple extra steps and ensured that only my local machine could get to that page, so folks on my local network couldn’t be popping up Things all day long by hitting the right IP in their browser. Other than that, I haven’t wanted to do anything fancier. That’s not really the point, after all.

Incidentally, I’ve found that limiting my consumption of distracting domains to the iPhone is a good way of regulating my time. If I have my iPhone in hand for more than a minute or two, I’m probably waiting around for something, in which case it wouldn’t hurt to see what’s up on Hacker News. Because reading on the iPhone isn’t optimal, I’m not likely to get lost in a sea of interesting links for an hour or two.

Switching Season

2009-08-10T00:00:00-07:00

Switching Season

Once, maybe twice a year, I do this stupid thing. At least I’m not alone in it, as friends get the same itch, but I have to do it. I think about switching away from the Apple platform.

The last time I did this thought experiment was last year, almost to the day. I agonized over whether or not Apple products were really simple enough to keep up with my minimalism fetish (verdict: not quite, but there’s nothing more minimal out there). Since then, I got a fancy new MacBook Pro at work, shelled out for an iPhone 3GS, bought my share of Mac software, advocated the joy of the Mac and iPhone to others, and have generally been a dutiful Apple citizen.

Until, of course, it was that time of the year again.

Mobile

This year, the impetus for Switching Season has been Apple’s much-discussed fall into authoritarianism on the iPhone platform, a distressing continuation of the behavior I wrote about last year. Of all the commentary on the subject, veteran Mac developer Steven Frank wrote the piece that rings most true to me:

“I’ve reached a point where I can no longer just sit back and watch this. The iPhone ecosystem is toxic, and I can’t participate any more until it is fixed. As people have told me so many times: It’s Apple’s ballgame, and Apple gets to make the rules, and if I don’t like it, I can leave. So, I don’t like it, and I’m leaving.”

This is how I’ve felt, politicized by Apple’s reprehensible corporate behavior. But, regrettably, embarrassingly, like a vegan caving to a steak dinner, I’m still using an iPhone.

I’ve tried alternatives. For my money, the only viable alternative to the iPhone is Android. Unfortunately, the experience of even the newest Android phones pales in comparison to the iPhone 3GS. I could relay my experiences here in detail, but they essentially mirror what Andre Torrez has documented of his own anti-iPhone experiment since last week.

The woeful performance and usability of Android is precisely the reason that Apple can treat developers (and consumers) however they like; it’s not even playing the same game, much less on the field with the iPhone. That Android is a mobile Linux platform is sadly apparent. Android suffers from the same issues that have plagued Linux on the desktop for years: the lack of integration between software and hardware, buggy and under-featured applications, a lack of attention paid to user experience issues. The encouraging openness and bits of innovation in Android are overshadowed by mediocrity.

When trying to use a myTouch 3G test unit that showed up in the office, a coworker walked by and asked, baffled, “are you still suffering through that thing?” I couldn’t figure out why, either.

Desktop

I have a similar love-hate relationship with the Mac desktop. On the one hand, nothing works better. There’s no better hardware, no more easy-to-use or reliable software. It’s simply the best personal computing experience available today. Which is all sort of frustrating for someone who likes to think about different ways of computing.

I’ve been considering violating a rule for computing happiness that has served me well to date: don’t fuss with having more than one computer. My desire for more work/life balance, and the general fever of Switching Season, had me considering buying a machine to keep at home, rather than using my company-provided MacBook Pro for both business and personal matters.

As I did last year, I researched the state of PC laptops. It’s still depressing. The “race to the bottom” started by the netbook trend has resulted in an even more barren landscape of ugly, cheap, underpowered hardware. Only Lenovo seems to be producing reliable, high-quality PC hardware, but the better units in their lineup are priced so as to be uncompetitive with the equivalent Apple machines.

Assuming I can justify the expense to myself, I could get a ThinkPad, familiar and homely and built like a tank. Then what? Run Ubuntu on it? Sure, Linux has evolved to the point that there’s not much tinkering required to have a functional laptop (if you do your research before purchasing), but it also boasts no marked improvement over OS X. I could run a tiling window manager and not have to fuss with manual software updates, but those niceties are traded for pervasive rough edges and inconsistencies, not to mention the loss of the near-seamless integration of the iPhone with iTunes, Address Book, and the rest of the Mac experience.

Considered, the process of switching away from OS X seems like an exercise in frustration. There’s just no demonstrably better way to compute right now. Adding a second computer to my life just opens up a world of synchronization nightmares. I’ve also tried periodic experiments with Emacs as a way of introducing a more open, extensible tool into my daily work without leaving OS X, but I can’t escape my general discomfort with the editor-cum-OS.

The True Meaning of Switching Season

All the above griping, while cathartic, isn’t really what this post this about. It’s about discovering why I lose myself in this obsession every so often.

Switching Season is about a desire to tinker, to play, to explore other possibilities for the tools that dominate my life as technologist. That’s why it comes on, strong and regular, grabbing at my attention and pulling me away from more measurably productive pursuits. It takes me back to age 14, installing Linux on a terrible old PC for the first time, trying to get things working, learning something new in the process. It’s about computer usage as a creative act, something that becomes harder and harder to experience the more proficient one gets with a computer.

Geeks who go through the same thing every year – and I know you’re out there – understand what I’m describing. The slickness of the Apple platform is at once brilliant and constricting, a sports car that even a veteran mechanic wouldn’t dare pop the hood of. It gives one the feeling that there’s nothing left to do because Apple has done it all. Most days, that’s exactly what I want, so I can focus on doing what Apple doesn’t. During Switching Season, though, I can’t escape wanting to do it all myself.

Searching for openness, simplicity, and a hackable sense of experimentation in the modern personal computing landscape is a fruitless endeavor, or at least one incompatible with also having a tool to get real work done. The more constructive thing to do, next Switching Season, would be to start experimenting with hobbyist hardware hacking platforms like Arduino, or something similarly low-level.

That’s what I’m going to try, anyway. And when Switching Season sets in for you, good luck.

Two Unfinished Ideas About The Future

2009-07-31T00:00:00-07:00

Two Unfinished Ideas About The Future

This is what I think about when I can’t sleep. I need these out of my head. And, somehow, they’re related.

Radical Transparency

What passes for transparency today is, in practice, mere translucency. Organizations reveal just enough information to avoid appearing closed and monolithic. They may attempt to communicate in a more human way, or provide open access to some of their data, but most remain fundamentally closed. Today’s “transparency” is little more than marketing.

That which is not open is secret. Secrets are vulnerabilities waiting to be exposed. Secrets are the illusion of competitiveness. Secrets are the bondage of serendipity. Secrets are lies of omission that eat away at our social and economic relationships. Their time is over.

The successful organizations of the future, be they states, corporations, communities, or collectives, will be radically open. Radically, unlike anything we’ve seen today. True transparency isn’t about a friendly company blog, or governments being slowly pried open with freedom-of-information initiatives. It’s about structuring society around openness, because openness is the only thing that’s sustainable.

You will see, in time, organizations that put everything on the table. Organizations for which there is no concept of non-public communication from day one; no internal email, nothing that isn’t a matter of public record, by design. Organizations for which every employee’s salary is public knowledge. Organizations that compete solely on the merits of their work, not on surprise, deceit, and manipulation.

People will think these organizations are crazy. And then, over time, radical transparency will become the norm.

Uncomputing

In order to enable the next monumental shift in the tech industry, we’re going to have to give up everything we know. The way we use computers, the way the Internet works, it’s all going to have to change. In fact, it’s all going to have to disappear.

The future of computing will be about the absence of computers as we know them. The step away from the desktop model of interaction will be the first down a long road of abandoning the ideas we’ve come to take for granted. First, the desktop will go. Then, the web. What remains is something wholly unclear, but incredibly valuable. It is the Thing That Comes Next. It’s a new market, and new markets are where the wealth and opportunities are.

Computing for the sake of computing – for entertainment, without a goal – will dissolve as a social practice. Technology will recede into the background of people’s lives; technology as a means to an end, not a means unto itself. Our current technology-obsessed mode is one of deep cultural exploration and experimentation, but it cannot last. It should not.

Something is happening at the fringes of the technology landscape. The related ideas of the Internet of Things and Ubiquitous Computing are one avenue, but perhaps not as humanistic as they need to be in order for this change to take root. Mobile computing also affords an opportunity to leave behind tired ideas of how technology fits into our lives, but the field is in danger of being dominated by dealers of the status quo.

The need for new social structures and new economic models will result in a reconception of information technology. Not just as a rehashing of paper filing systems and abaci and ledger books, but something wholly other. The changes we witness daily – for example, the emphasis on “social computing” of the last several years – blind us to broader possibilities.

Stay open to the idea that the future is unrecognizable. You will come to know it.

Fever and the Future of Feed Readers

2009-07-18T00:00:00-07:00

Fever and the Future of Feed Readers

Time was, every self-respecting geek lived and died by his feed reader (or aggregator, if you prefer). Just several years ago, the number of subscriptions in your RSS-chomping tool of choice made for bragging rights. “200? Oh, I can get through 500 feeds a day.” More subscriptions meant you were more in the know. Really good lists of subscriptions were traded amongst friends, but cautiously, just as one might hold back a recommendation to a superb but little-known restaurant.

At the time, the only real debate was around the best way to present all this information. Some preferred a river of news, others preferred their content categorized and neatly filed, like sections in a newspaper. But everyone was in agreement: having all this fresh content collected for you in one place was a boon. It was a change in mindset, and it seeded the demand for what is now being called the Real-Time Web. (Incidentally, the Real-Time Web is next year’s Web 2.0. If you’d like to appear cool and aloof, start disdaining the expression now).

Today, at least in the web-tech echo chamber, feed reading is quickly falling out of fashion. Too many sites producing too many feeds of dubious quality means information overload, and a creeping sense of obligation to keep up with a torrent of questionably relevant content. Some have gone back to checking a handful of bookmarked sites, as we did in the early days of the web. Others rely on social aggregation sites like Reddit, Digg, and Hacker News to show them what’s worth reading. Both strategies are highly manual and, to me, distressingly unoptimized.

Abdicating Aggregation

Another camp all but eschews the idea of trying to keep up with feeds. Chris Wanstrath, co-founder of the superb social coding site GitHub, is one of the more visible advocates of this approach, saying in a tech conference keynote last year:

“Stop using Google Reader or NetNewsWire or whatever the kids are using these days. It’s not worth your time. [L]et other people do the filtering for you. Use your time for other things.”

This statement initially rings true. We’re in the age of social networking, after all. I’ve told social sites about my friends, and my friends are always talking about things, so just show me what my friends are talking about and I’ll always be in the loop, right? Then I can focus on my own interests and projects. Sounds great.

The problem with abdicating your content consumption to other people, though, is other people. Perhaps it’s overestimating my ability to find interesting things to read, but I don’t trust my friends and the Internet at large to educate and entertain me. In the venn diagram of my interests and my friends’, there may be 80% overlap, but most of the content that I’m going to find deeply engaging is probably in the leftover 20% at the margins.

There’s also a sort of collective danger to the strategy of exclusively consuming information through social osmosis: if everyone does it, who’s going to find the interesting stuff? Who takes the reigns as the editors, the arbiters of taste? Going back to a post I wrote in 2003, who will be our cool shit aggregators?

If everyone took Wanstrath’s advice, nobody would do any filtering and nobody would consume anything. Realistically, we’re in no danger of that, but we’re also not seeing a radical improvement in the way we consume information on the web. Surely someone’s investigating another strategy?

Blending Subscriptions with Social Data

Google Reader is, as evidence of the slowly dying field of feed reading, pretty much the only regularly-updated, widely-used aggregator left on the web. Bloglines has been gasping for air for over a year, and NewsGator is positioning itself towards the enterprise, presumably trying to scrape some money out of the generally unprofitable business of aggregation.

Reader has been something of a playground for Google, and one of the products for which the behemoth has been most responsive to public feedback. When Reader launched, its interface was nigh-unusable. It was updated, improved, and gradually became the only feed reader worth using — and not just on the web, something it pains me to say as the owner of licenses for multiple desktop aggregators that eventually had their price driven down to free, and have since seen little attention from their developers.

Today, Google seems hellbent on cramming its otherwise clean and speedy products with cumbersome, poorly conceived social features. Presumably they see social networks as a threat to their valuable side business of, uh, completely free products, and this is their ham-fisted response. In Reader’s case, the user response has been one of confusion and derision.

Seeing content filtered through my social lens seems like the marriage of traditional feed reading to Wanstrath’s more osmotic approach. Reader’s implementation doesn’t prove this to be a happy union. The tool is now cluttered with smilie faces indicating content that my friends liked, only Google has fairly incomplete view of who my friends are because they’ve yet to create a social experience that encourages me to share that information. Reader’s myriad competing ways to share, vote on, annotate, and remember items further detract from its former appeal.

I’ve given up on Reader, but I’m not ready to give up on feed reading just yet. I wanted to try one more experiment.

Enter Fever

Fever is a feed reader designed and built by Shaun Inman, the developer behind the popular Mint web traffic analytics product. Like Mint, Fever is $30 (USD) and runs on your server — a ballsy proposition in an age of free software running in the proverbial “cloud”. It is unapologetically for power users.

Fever’s proposition is straightforward: supply it with the feeds you always want to read, and supplement those with feeds that you only want to read the juicy bits of. Fever will then show you a sort of personal Techmeme or Google News, pulling together stories that reference common URLs. Fever’s precise formula for this isn’t discussed on the product’s relatively curt homepage. Take it or leave it.

I forked over my money, spun up a virtual server, and have been using Fever for several days now. Installation was as straightforward and slick as you could hope for given that Fever is a self-hosted web application. Special features aside, it handles the basics well — imagine Google Reader before all the social bloat and with a far more attractive design. Fever’s design is not perfect, but it’s easy on the eyes and pleasant to use. Put another way, Fever doesn’t make it harder to read feeds much as you always have.

The $30 question, though: does Fever really float the best, most relevant content to the top in a personalized way? Can it dig through all the noise on the web and show you what you need/want to know at a glance? The free answer: sort of.

For starters, it’s easy to pollute your corpus of “signal” feeds, which Fever calls sparks. Fever needs sparks that contain a lot of links. If you put “top” feeds from Digg, Reddit, and the like into Fever, you’ll basically just end up with your own dim, mostly irrelevant slice of the web. Fever really needs folks like Waxy, Laughing Squid, and Trivium to keep churning out link blogs full of references to good content. Without those sort of quality, URL-rich feeds, your Fever’s view of what’s hot is going to be lukewarm.

For this reason, Fever is just fine for floating good techie content to the top, but poor for most any other subject. I’d love it if Fever could find me good posts from the set of minimal techno or cocktail blogs I subscribe to, but link blogs – and, indeed, linking outside one’s own site – just aren’t as prevalent in those communities. Fever did similarly poorly given a number of sparks for top world news; a paucity of URLs means Fever can’t replace Google News for figuring out what’s on the front pages of the world’s newspapers.

It’s disappointing that I can’t depend on Fever to be a one-stop shop for my daily information intake. With my current heavily-curated collection of subscriptions, I can rely on Fever to be a sort of no-bullshit Techmeme, but little more. For the topics of world news, music, art, culture, humor, food, and drink, I still need to read a number of feeds entry-by-entry.

Given Fever’s initial cost, plus the ongoing cost of hosting a server on which to run it, I can’t imagine that it’s a tool that will last long in my tool belt. I already regret the time I spent setting it up and tuning my feeds, and I can’t really justify keeping it around for the sole purpose of being a less-encumbered Google Reader.

The Future of Feed Readers

I’m not sure what the solution is here. Feed readers as we’ve known them are dying, but it’s as yet unclear what will take their place. Filtering feeds for relevance algorithmically seems all but fruitless; filtering through the social graph is only a slight improvement, but misses the rare content that may only strike a chord with a small audience.

If there’s one thing I’m convinced of at the end of this exploration, it’s that there’s more work to be done, and more businesses to emerge in this field. Social networks alone aren’t focused enough tools to bubble up and share quality content. My hope is that a surplus open data of the sort we’re trying hard to share at Twitter will help spawn a new generation of tools to manage the flood of content. I don’t think it’s a problem that Twitter, or any other pipeline for information, can solve on its own.

With all that said, perhaps the right approach really is to abdicate one’s consumption of content to whatever you’re passively exposed to, and to occupy your mind with other things. The act of creation is almost always self-affirming, and the act of consumption so rarely is.

The Tapir Book

2009-07-07T00:00:00-07:00

The Tapir Book

When my coauthor and I began working on Programming Scala last year, the most frequent comment I got was, “you’re never going to want to write another book once you’re done.” Well, as of this morning, I’m pretty much done, and I can say that I’d very much like to write another book. I don’t think, though, that I would commit to another one while working a full-time job, and I certainly wouldn’t recommend trying to write a book while working at a rapidly growing startup.

When I announced that I was going to be working on the book, a critic insinuated that it would detract from my work at Twitter. In practice, the opposite was true. I had many an evening or weekend afternoon of writing interrupted by fires at work that needed fighting. For that reason, I wouldn’t consider taking on another book project until I can give it my full attention. The value of sabbaticals as practiced in the academic world was made crystal clear to me throughout the writing process.

There’s a reason why Dean’s name comes first on the book’s cover: he is unambiguously the principal contributor to Programming Scala. We worked from his outline, and though much was changed and reorganized collaboratively, the backbone of the book is absolutely Dean’s. I couldn’t have asked for a better coauthor. If anything, I wish that I could have contributed more equally to the text. What I did contribute, though, I’m proud of. We handed off our content to the production team at O’Reilly earlier today, and I can hardly wait until October to have the final product in my hands (and on my Kindle).

I took on the book in part to develop a mastery of Scala, and I’ve looked forward to learning something new every time I sit down to write, week after week. Though I understand more of the language than I did when I started, I still don’t feel that I’m on the level of folks like David Pollak, Jorge Ortiz, Daniel Spiewak, and the rest of the Scala gurus who dove into the language well before Dean or I. Still, it’s been an incredible learning experience, and I’m extremely grateful to everyone who made it possible, not least of all our editor, Mike Loukides.

This coming weekend will be my first in many months that is completely open, free of examples that need to be written, sections that need to be reorganized, reader feedback to incorporate, and unfamiliar concepts that need to be researched before I can write about them with confidence. With the free time I now have again, I can get back to writing open source code, mixing music, exploring the Bay Area, posting to this blog, and all the other things I’ve set aside while working on the book.

It’s liberating to be done, but bittersweet.

Since I Left You

I have managed to do a bit of this and that while wrapping up the book. We’ve set up a blog for the Twitter platform, and I’ve contributed a few posts to that. I’ve managed several talks, and committed to a couple more later this year.

Most cathartically, I’ve laid open my virtual notebook of ideas for all the web to see. I’m going to be at Twitter for a couple more years at least, if all goes well, and it seems a shame to let ideas sit in a text file and rot in the interim. Some of the ideas I fully intend on seeing through myself, in time. Others I hope will be picked up, improved upon, or made irrelevant by people smarter and more talented than myself. Either way, it’s been an experiment in radical openness (a topic I’m preoccupied with), and one that’s paid off from day one.

It should be less quiet around here, again, now.

Reading The Web on Kindle 2

2009-05-17T00:00:00-07:00

Reading The Web on Kindle 2

When I ordered a Kindle 2 shortly after the device was announced, I promised that I’d review it. Thanks to the book, I haven’t had time for much personal writing, but I’m sneaking this in today between edits.

The Kindle works so well that it’s difficult to review it in any critical sense. It delivers the experience you’d imagine it does: you buy books and periodicals, they show up instantly on the device, you read them. Text looks great, it’s easy to use, and it does indeed have the widely reported effect of making you read more. I had to suffer a couple of lemons before I got one that worked, but Amazon was courteous and quick about replacing them.

Really, the Kindle is a no-brainer purchase. Go buy one; you’ll be happy with it. That is, if you only want to read books and periodicals on it.

Dun-dun-dunnnn.

The Kindle is great for traditional media, but I wanted to see how it fared with hypertext. Shortly after I got my Kindle, I stopped using Google Reader. I dumped my feeds into Kindlefeeder, pruned out the purely visual and aural subscriptions, and set up a delivery schedule such that I had a fresh digest waiting for me when I got home from work. When I found things during the day that I wanted to read in depth, I’d pop them into Instapaper and they too would be batched into a digest and delivered once a day.

For a while, this worked well. I was no longer tempted to check the feeds or read interesting articles during the work day, as I had no convenient way of doing so. I got into a routine of reading my daily Kindlefeeder digest just before bed and going through the Instapaper items on the weekends. Looking at text on the Kindle is such a vastly improved experience over reading on a computer or iPhone that I forgave the occasional formatting or delivery error.

If the hypertext you read doesn’t make use of links, pictures, movies, block quotes, or essentially anything other than paragraphs and markup for emphasis, you’re set. In my experience, though, the Kindle is a pretty lousy platform for reading hypertext. The browser is filed under “Experimental” for a reason: the web looks weird in it, and the browser doesn’t behave quite like the rest of the Kindle. Bouncing between a Kindlefeeder digest and the browser is slow and clumsy. Technical articles I’d saved via Instapaper lost critical formatting and diagrams. Frustrating, all around.

This is nobody’s fault, per se, certainly not Kindlefeeder’s or Instapaper’s. Amazon is pretty clear about what you’re supposed to use the Kindle for, and that’s books and magazines and maybe the very occasional text-only blog. The Kindle is awesome for these things. It’s just not a multimedia device.

“Well hurr”, you say, but I guess I hadn’t realized how many of the feeds I read are composed of a rich mix of text, audio, video, and still images. I’ve been doing the blog thing for long enough that I remember when feeds with images were pretty unorthodox. Tumblelogging has really opened the world of syndicated online publishing up to more than just text. That’s a good thing for the web, but a bad thing for constrained devices like the Kindle.

So, after a couple months of reading web content pretty much exclusively on the Kindle, I’ve ultimately decided to go back to Google Reader and a browser on my laptop. Doing so requires more self control, and text doesn’t look as good, but it opens back up a world of multimedia content that, much to my surprise, I had come to miss.

In a way, I prefer the idea of a barrier between my Kindle and the web. I want my reading time to be about deep engagement with substantive content. Some blogs qualify for that time, but not many. I’d rather flip through my feed reader over my morning cup of coffee and keep the Kindle for novels, The New Yorker, and perhaps an academic article or two.

Mending The Bitter Absence of Reasoned Technical Discussion

2009-04-04T00:00:00-07:00

Mending The Bitter Absence of Reasoned Technical Discussion

There’s a counterpart to my post on technology journalism that I’ve been hesitant to write. Just as most professional journalism on high technology fails us today, so too does the online discussion amongst technologists as a community.

Social media (blogs, community news sites like Reddit and Hacker News, Twitter and such) have swept in to fill a vacuum between peer-reviewed academic journals and water cooler conversation amongst software engineers. Anyone with a blog can publish development war stories, benchmarks, or an interview with another developer. It’s a world of engineer’s notebooks laid wide open, and in theory, we should be more informed as a profession than we ever have been.

In practice, the conversations that are most widely heard in the tech community are full of inaccuracies, manufactured drama, ignorance, and unbridled opinion. In discussing these Internet-spanning debates with non-technical friends, comparisons to Hollywood tabloids come first to mind. It’s a time sink for an industry that should be a shining example of how to use the newest of media for constructive debate.

Faith and Numbers

My first harsh exposure to this sort of discussion was in 2007, several months into my contract on Twitter. I was asked to do a short email interview about our system and its development. I asked the interviewer if he’d prefer to interview the more senior developer on our team, but he insisted that he wanted an “in-the-trenches” perspective. I answered his questions to the best of my ability, and went back to work.

Imagine my surprise a few days later when my answers to a series of straightforward questions on a blog were the talk of the web application development community. One comment that seemed particularly uncontroversial to me was at the core of this nerd firestorm: Ruby is slow.

For the next several days, I could think of only one thing: why is something you can measure controversial? Are we not engineers? Is this not Computer Science? Why are we discussing the performance of software as if it were a matter of faith, or opinion, or preference? If you disagree, simply run an experiment. Did we somehow lose our Enlightenment values on the way to Web 2.0?

Of course, what I had said about Ruby was of no surprise to those who had been working with the language in earnest. In a private IRC channel, I was told by the Rails elite that I’d said nothing they didn’t all know. It was simply a tradeoff: work with a language you enjoy, accept the slower performance, buy more servers. Unless you have some deep insecurity about your choice of programming language, there’s simply no reason to be offended by such an assertion.

Time melted away my frustration, Twitter continued to grow, and I eventually got a chance to speak in person with some of the most vicious responders to my little interview. Amazing how suddenly reasonable and transformed a person can be when they’re forced to look you in the eye.

Two Years On

Earlier this week, I gave a presentation at the Web 2.0 Expo in San Francisco. As I made clear at the beginning of my talk, I was presenting in part because I’m co-authoring a book about the Scala programming language, one that’s being published by the same company that puts on the Web 2.0 Expo, O’Reilly Media. Cross-promotion, I believe it’s called. Scandalous.

For me, writing the Scala book and giving these talks are a labor of love. I’ve given up my free time to deliver the book, just as my co-author has taken time out of his personal life and consulting schedule. We’re getting paid for our work, but it’s no secret that writing a technical book is not exactly the fastest route to staggering wealth. I don’t get paid to speak; at best, my travel expenses are covered. I speak and write because it’s a way I can contribute to the Scala community, and I want very much to share the brilliant work they’ve done with anyone who will read or listen.

My presentation argued that Scala is a great choice of language for the next wave of Web 2.0 businesses, which must operate in a far more restrictive economic climate than their predecessors. My post on recession engineering sums this idea up. I said that people chose languages like Ruby and Python for early Web 2.0 businesses because those languages are a pleasure to develop rapidly in, and that Scala has that same quality while delivering better performance and leveraging the assets of the Java Virtual Machine, which can translate to dollars saved on hardware and new development.

I went out of my way to explain where Ruby’s strengths are as we see them at Twitter, without spending so much time on the issue that it detracted from showing off Scala’s features and making the business case for the language. I briefly covered why we didn’t make use of other languages. I did not discuss why nobody should ever use these other languages, but simply why we (Twitter) currently do not. It would be absurd and disrespectful to tell a room of other engineers why their choice of language for their projects – projects they know everything about and I know nothing about – is wrong. I try to speak only about what I know first-hand.

That a language is “fun” or “agile” or “beautiful” is unmeasurable and fuzzy. That a language is fast in comparison to another language, or can make use of code written in other languages without penalty, is not. Those incontrovertible facts, coupled with Scala’s specified and tested features, were the crux of my argument. Nobody who was actually present at my presentation found these facts controversial enough to stand up and dispute them. Indeed, the reaction amongst attendees seemed to be one of newfound curiosity in Scala, which is all I was trying to accomplish.

Some hours after I’d left the conference, the dual-headed distortion machine of the tech press and social media went to work. Before long, the story about my presentation had gone from “Scala is a nifty language and you should think about it for your business” to “Twitter engineer spits on the grave of Ruby, exalts Scala as shining new deity”. Time warp to 2007, numb and dismayed.

How To Have A Reasoned Technical Discussion

We’ve come to accept that trying to have a reasonable discussion on the Internet is like insert any number of increasingly offensive metaphors here. Usenet, IRC, forums, blogs, and now media like Twitter have all been black-marked as houses unfit for reason to dwell within. And so we roll our eyes, sigh, and quietly accept the idiocy, the opportunism, and the utter disrespect for our peers and ourselves that is technical discussion on the Internet.

This need not be the case. It is possible to have a reasoned technical discussion on the Internet. People do it every day, particularly in smaller online communities where social norms are easier to enforce. We can do it.

The next time you’re thinking about engaging in a technical discussion, run through these questions before you hit the “post” button:

Are you responding to facts? With facts?
Have you read any primary source materials on the issue you’re discussing?
Do you have any first-hand experience with the technologies or ideas involved?
Do you have any first-hand experience with those technologies operating at the scale being discussed?
Have you contacted the individuals involved in the discussion for further information before making assumptions about their findings?
Are you falsely comparing technologies or ideas as if there was a zero-sum competition between them?
Are you addressing your peers with respect, courtesy, and humility?
Are you sure that what you’re posting is the best way to promote your self, product, project, or idea? Does it demonstrate you at your best?

Et cetera, et cetera. Or, essentially, a brief reintroduction to logical thought and civil society.

And a final tip:

Some technical discussions veer towards the purely aesthetic. Thankfully, the humanities have provided us tools for reasoning about that which hard science may not be able to measure. Spend some time with art and theater criticism and you’ll find intellectual instruments aplenty for the comparative evaluation of seemingly intangible qualities such as beauty, theme, and emotion.

Conclusion

I don’t get much out of writing this sort of proscriptive, parochial content, but it’s my honest hope that someone will read the above, stop, think, and make a better contribution to the discourse of the technical community.

This mode of discussion is lower than us. Yet, lest we make an effort to rise above it, it’s all we deserve. Don’t waste your life screaming into the void. Make things, measure them, have reasonable and respectful conversations about them, improve them, and teach others how to do the same. Set emotion aside, and think how much we could accomplish if we had the humility and grace to learn from our peers.

Towards Better Technology Journalism

2009-03-03T00:00:00-08:00

Towards Better Technology Journalism

Rarely does technology journalism produce informed, correct, relevant, and readable content. This is a sorry and damaging state of affairs.

I’ve been drafting this post in my head for ages, and bringing the topic up to friends and colleagues ad nauseam. One approach I could take is to rantingly provide example after example of miserable technology journalism. For anyone immersed the culture of high tech – that is, those of you who care about this issue and have read at least this far – those examples are practically unnecessary.

Most technology professionals I know roll their eyes at our industry’s press. “What are you going to do? Can’t live with ‘em, can’t get publicity for new products without ’em” seems to be the mindset. To ask for truly superb coverage of anything more than the latest gadget is asking too much in today’s tech media landscape. As an engineer, a consumer, and an avid reader, I’m unsatisfied with this.

Before I dive in, I’d like to clarify that this is entirely my personal opinion, and in no way reflects that of my employers. To maintain focus in my job, I ignore any and all press about Twitter that I’m not forced to read while strapped to a chair with my eyes pried open, Clockwork Orange-style. My personal projects have been covered with reasonable accuracy. This is not axe-grinding. Moreover, my agenda is not about ensuring that technology business and research gets a pass from the press; if anything, our industry should be regarded more critically.

With that said: instead of harping endlessly and unproductively on the culprits, I’ll briefly feature two recent examples of inept tech reporting. Then, I’ll expand on the problem and offer some potential solutions.

Example One: “TechCrunch Are Full Of Shit”

That phrase has been a rallying cry in the web community of late, urged on by a post on Last.fm’s blog. Long story short, the popular web-oriented technology news site TechCrunch reported on a rumor, something the site does seemingly as standard operating procedure. Generally, companies and individuals don’t bother to retaliate when slandered by TechCrunch, as to do so would lend an iota of legitimacy to to the site, while reducing the victim to their level of pettiness.

Last.fm bucked this informal policy and took a stand. They were quickly validated for doing so. The damage to the company’s reputation, though, is done. In an industry where ending your consumer relationship with a company is one click on a “delete my account” button away, misleading and false press can utterly undermine a business.

TechCrunch hardly count as “journalists” or “the press” by any reasonable definition. They’re a tabloid masquerading as a legitimate news outlet, a sort of Drudge Report for nerds; they lack even the sense of humor of actual tech industry tabloids like Valleywag. While TechCrunch may have started out as a blog, free of the restrictions and expectations of traditional journalism, their content is now syndicated by the Washington Post. TechCrunch’s is one of the most widely-heard voices in technology reporting. This should be considered an embarrassment to our industry.

Still, to really illustrate what’s wrong in technology journalism, an example from a credible publication is necessary.

Example Two: Dirtying a Clean Slate

In mid-February, the New York Times published a piece by John Markoff with the provocative title Do We Need a New Internet? The article discusses a fascinating research project called Clean Slate, set to “reinvent the Internet” with an eye towards security. Anyone with some network engineering experience understands that while the Internet is a clear success on a human scale, there’s room for improvement at the level of bits and bytes, particularly when it comes to security.

Markoff does this essential research project (and credulous readers alike) an enormous disservice by veering away from actual reporting at the end of his article. The last several paragraphs are nothing more than speculation on the author’s part, and not even speculation that’s of particular relevance to the aims of the Clean Slate project. Beyond being generally in the ballpark of security stuff, there’s nothing pertinent there. Though its placement in the “Week In Review” section may account for the editorialization, it’s serious subject matter being treated carelessly.

While not outright slanderous, the New York Times have not, to my mind, fulfilled their journalistic responsibilities when discussing Clean Slate. Airy speculation is for blogs (cough) and barrooms. No wonder that security experts like Ben Laurie were aghast at the irrelevance and incorrectness of Markoff’s conclusions.

The poor quality of technology journalism is not simply an infection plaguing unaccountably popular blogs. It’s real and present in the most trusted names in news.

The Problem

The scary truth of information technology is that it’s just too huge a domain to be an expert in, even if you’re a full-time engineer. I’d wager there’s just a handful of people on this planet who can claim expertise in everything from silicon up to human-computer interaction. Even if most engineers were halfway-decent writers, most engineers aren’t equipped to write about technology in the large.

The majority of technology journalists are even less equipped. Many have no engineering background. They’ve never built anything like the things they write about. Or, if they were once engineers, they haven’t written a line of code or soldered a circuit in years. In a fast-moving industry, professional engineers get left behind the state of the art all the time. How can journalists without any engineering expertise possibly hope to keep up? Simply tapping expert sources isn’t enough. A reporter can’t simply string together quotes from PhDs and CTOs and end up with something cogent, accurate, and informative to a non-technical reader.

We shouldn’t be content to trust the public record of high technology to individuals ill-equipped to report on it accurately. In an age where new media have enabled the people who make technology to produce a dynamic record of its creation and use, the role of the technology journalist is to tell a story that reaches outside our industry and community. It is, then, partly our responsibility as an industry and as a community to ensure the quality of that shared story.

The worth of accurate technology journalism produced by qualified professionals is unquestionably high to the technology industry and the public it serves. How do we fix this?

Solutions

This is not an easy problem to solve. Journalism of all sorts is under attack; the lack of quality reporting and the corresponding lack of trust and engagement amongst readers is not unique to the technology industry. High tech has the advantage, though, of a track record of interdisciplinary problem-solving. Solutions are out there.

Here’s several, to get the ball rolling:

Teach technology reporting in J-school (better) — Journalism schools offer specialized tracks for business reporting, financial reporting, even sports reporting. There are a handful of programs in technology journalism, but these tend to focus on the hard sciences. High tech industry professionals should help instructors develop accurate and informative curricula, and make themselves available to speak directly to journalism students in the classroom. Want someone to talk to your journalism class about how web applications work? Email me. I hope other engineers will contribute their expertise to the classroom.
Report on your 20% time — An increasing number of tech companies encourage side-projects and open source contributions. Why not use that time for journalism as well? Let’s get working engineers in the field, reporting on the subject matter they know better than anyone. This would require a strong editorial hand, but the risk seems worth the reward of highly informed technology reporting.
Incentivize technology reporting as a career — It’s hard to make a buck as a technology journalist, particularly one who reports on something more substantial than gadgets and empty enterprise software press releases. No wonder that TechCrunch has gone the route of sensationalism; it drives ad clicks and sparks debate, making a potentially dreary beat profitable and exciting. Tech journalism isn’t sexy, but it could be made so. That change starts with breaking the cycle of low-quality tech reporting, giving prospective technology journalists a set of role models they can aspire to.

Your solutions are needed. Without them, the only hope for technology journalism is that communication channels like blogs and Twitter outpace the inaccuracies of bad reporting with the distribution of fact. To a degree, this is already happening: friends only heard about the Last.fm scandal because the corrected story was making the rounds on Twitter, “routing around the damage”. This is small comfort, though, and a shallow goal for our industry and community.

You can assume the inevitability of mediocre technology journalism, or you can contribute solutions and make a change. The fidelity of the public history of high technology is in your hands.