Beard's Eye

The Code That Needs To Be Written

noreply@blogger.com (Anonymous) — Wed, 12 Jun 2013 20:16:00 +0000

I have many good reasons for not wanting to be a manager, but they all boil down to this: I enjoy writing code.

I recently read a blog post about not writing code. If you haven't read it, I encourage you to do so. It's insightful and right on all counts. It's also biased, as you can see from the following quote:

My job isn't to write code. It never was. It has always been to make people's lives better and more productive.

Years and years of experience have taught me to avoid writing code when I can and write as little of it as possible. Every line of code you write has its cost. Some of that cost is the time it took you to write it. Some of it is the time it took you and, hopefully, at least one more person to test it. And then there's the time it will take you or someone else to maintain it. All in all, writing code is not something to dive into without careful thought.

As a coder, every line of code you don't write spares you and your team time. The question is, how do you spend that "extra" time? You could deliver the product earlier! Just kidding, it's more likely you'll spend that time writing other, more important code.

All joking aside, the above is a very serious, important point: don't avoid the code for the sake of avoiding code. Write the code that needs to be written. And there's always some code that needs to be written. While it's theoretically possible to build a complete, non-trivial software solution without writing a single line of code, I've yet to see that actually happen. At the end of the day, when all the management decisions have been made, someone will probably have to write some code

There are many ways of making people's lives better and more productive. Writing code might not be my business, but it is my calling.

Off With Their Heads: The PyCon Incident and Our Society

noreply@blogger.com (Anonymous) — Thu, 21 Mar 2013 14:03:00 +0000

Fair warning: If you've come here looking for a quick validation of your arguments for or against Adria Richards, you can go on elsewhere. I'm not interested in "winning a debate" here. Also, I'm a white male, so if you think that makes me unfit to have any opinion on this topic, I also invite you to move on. Still here? Good, let's have a civilized conversation.

If you're part of the software development industry and have looked at the social media over the past two days, you'll have heard of Adria Richards and what happened at PyCon. Just in case you haven't, though, here's the short version: two guys working for some of the sponsors were sitting within the earshot of Adria and making crass jokes involving "forking the repos" and "big dongles". I invite you to read Adria's blog post about it before going on.

Right off the bat, let me tell you that I'm not writing this post to condemn either these guys or Adria. Like I implied at the top, I don't see things black and white here and I think that too many people currently participating in the conversation -- including Adria -- are looking at this as a battle to win. This is something that bothers me a lot and I think something needs to be done about it. Before we get to that part, though, we need to discuss the incident a bit.

What Happened?

The first I heard of this incident was this very morning, when I looked at Hacker News front page and noticed that there was a story that had over 900 points and more than 800 comments. It was a link to a Pastebin text that paints what turned out to be a rather one-sided picture. My first reaction was outrage: just one paragraph into the text, my imagination conjured a vivid scene of two guys talking in a hallway and someone sneaking up on them to take a photo. I instantly developed a strong dislike for Adria and had to fight that dislike all the time while I was looking for more information. There's still a part of me that wants to hate Adria and I say that to illustrate the importance of not acting out on your first impression.

Here's the thing you need to keep as first and foremost in your mind whenever you stumble upon a situation like this: you don't know enough. I don't know what the words of the joke were. I don't know if their joke had anything to do with what was being discussed on stage or not. I don't know if the guy that got fired had a stellar record within his company or was considered problematic. Those are just some examples of things the vast majority of us don't know even now.

So let's start by sticking to what we do know:

If you're at a convention, you represent your company and your actions can reflect on it.
If you're sitting together with a bunch of people arranged in rows of seats, your expectation of privacy drops rapidly.
It is not acceptable to ruin someone else's experience for them with your conversation. It's like talking in a movie theater.

All in all, it was definitely not okay for the guys in question to do what they did and neither Adria nor anyone in her position should be expected to tolerate it. For the record, I also agree that just talking to the guys directly would most likely been ineffective. I've seen people deflect the complaint too often to keep believing that this approach works.

That said, things turned ugly on more than one level and this is what I want to discuss here.

Off With Their Heads!

If you look at Adria's Twitter stream and at some of the blog posts around, you'll notice something interesting: people have taken sides and are arguing against the other side. It's like we've dug two huge trenches and people in each are now taking potshots at the other trench.

Let's take a step back and look at the outcome of the incident, without foaming at the mouth. One good thing is that the PyCon Code of Conduct got upheld and it's been made clear once again that those aren't just words. Another good thing is that unacceptable behavior, got punished.

On the other hand, one undeniably bad thing that happened is that a person lost a job. I say "a person", because it should not matter whether it was a man or a woman or what their skin color is. I agree wholeheartedly with Avdi Grimm when he says:

We need space between "you're fine" and "you're fired".

Did the guy from PyCon deserve to be fired for his jokes? It depends on a number of factors: whether his joke was downright sexist or just immature, whether he was fired just for that or he had other problems at work, and so on. If we assume -- as most people seem to -- that his jokes were merely immature and that he was fired only for that, then I would argue that such a drastic disciplinary measure was out of proportion.

Another bad thing that happened is that a huge controversy has been made out of this and the rift between men and women in tech -- a ridiculous rift that shouldn't exist in the first place -- has likely been made wider.

Nearly everyone is out for blood. Some people are viciously reveling in the fact that one of the guys lost his job, while others are furiously demanding an apology from Adria and SendGrid.

It's like we fell down the rabbit hole and everyone is either the Queen of Hearts, screaming "off with their heads", or a part of her army of card soldiers.

It's too easy for things to spiral out of control and out of proportion and I find it worrying that, in our society, you can cause a social media avalanche with one tweet and set loose a mob on someone. Regardless of whether you feel righteous about it, you should step back and consider all the possible consequences before committing to an extreme action.

Punishment, Fear and Empathy

Trigger Warning: this section makes references to a high-profile rape case, so you might want to skip to the notice that says "End Trigger Warning".

Events at PyCon are really just a reflection of our society's attitude in general. One recent example that illustrates this attitude is the Steubenville rape case. In case you're not familiar with it, go Google it, because I'm not going to describe any details here. What I do want to focus on was the way certain media comments were handled.

CNN reporter Poppy Harlow made the following comment about the case:

It was incredibly emotional, incredibly difficult even for an outsider like me to watch what happened as these two young men that had such promising futures -- star football players, very good students -- we literally watched as, they believe, their life fell apart.

The public reaction was one of outrage. Again, my first reaction was similar. However, I later stumbled upon an honest and humble questioning of this reaction by a writer I admire greatly and that made me stop and try to think more critically.

There is no question whatsoever that these two youths deserve their punishment. I believe there's no defense or justification of their acts. That doesn't mean that the situation isn't tragic, not only for the rape victim, but also for them. These are two people that could have had a much better future, if only we, as a society, could have detected early the flaws in their upbringing and corrected them. It really is tragic to see all three lives damaged so profoundly. That's why we call certain things tragedies: the net outcome is negative.

Yet the society at large seems to value only the punishment and no one is allowed a modicum of human compassion and empathy for the bigger picture.

End Trigger Warning

The problem with the PyCon incident and with our society in general is that we manifest disproportional care about punishments. It's certainly necessary to punish inappropriate behavior, but is it enough?

As a father of a six year old kid, I can emphatically claim that it isn't. Believe me, it's a mistake I made earlier and the result was that my son was afraid of me. When he said so, my heart nearly broke. I've been careful ever since not to repeat the same mistake.

Another thing I had to learn was that not every offense deserves the same punishment. If your kid peed his pants because he didn't want to pause the damn video game, it's not the same thing as throwing a temper tantrum because you told him he can't play before he does his homework.

You cannot solve a behavioral problem by simply punishing the bad behavior and stopping there. You need to look at the bigger picture and you need empathy. Fear is a poor problem-solving tool.

I applaud Adria Richards for not tolerating unacceptable behavior and standing up to it, instead. I also disagree with her methods and believe there were better ways of dealing with things.

Most of all, I strongly feel that, instead of glorifying the outcome of the PyCon incident, we should focus and keep looking for ways to change our culture, so that the history doesn't repeat itself.

UPDATE #1 (2013-03-21, 15:02 CLST): Continuing the trend of punishment-oriented overreaction, it appears that SendGrid has decided to fire Adria Richards. I am still hoping that they were simply hacked and that this is false information spread by the misogynist mob of script kiddies, but that hope is fading fast. I've never been more depressed about people illustrating my point for me.

UPDATE #2 (2013-03-21, 19:29 CLST): SendGrid has just posted an official explanation of their decision. While I still disagree with two people losing their jobs over this whole situation, I wholeheartedly agree with SendGrid's statement that Adria's "actions have strongly divided the same community she was supposed to unite".

Mrs. Mayer's Major Misstep

noreply@blogger.com (Anonymous) — Sun, 17 Mar 2013 15:24:00 +0000

I can't help feeling that a lot of us have had some rather wild expectations from the 21st century. We watched "Back to the Future" in our teenage years and fantasized about flying garbage-powered cars. We read Asimov and fantasized about having robots throw out our trash while we colonize other planets. We watched Star Trek and the idea of a post-scarcity society filled us with hopes and dreams.

The reality has yet to catch up with science fiction, but we did get to see quite a few technological advances that were nothing short of amazing. Unfortunately, it seems we expected to see the society change for the better at an equal pace as technology and that's why we keep getting disappointed on a regular basis.

Take telecommuting, for example. The very idea that you can be in a large team doing a complicated job and contribute your part from home would have been considered fantastic when I was a kid. Indeed, it was a fantasy back then: Asimov's novel "The Naked Sun" describes a society where almost every social interaction is done via holographic telepresence. Since then, the availability of telecommuting as an option has improved many lives and quite often resulted in a net gain for everyone involved.

Then came Marissa Mayer's decision to ban telecommuting at Yahoo. Unsurprisingly, the initial reactions across our virtual global world were largely those of outraged criticism. A furious debate sprung up between those who condemn her move as oppressive and those who see in it a logical necessity for Yahoo as a company to catch up with the competition. More than two weeks later, the discussion has slowed down fractionally, but it's far from stopping, as new articles and posts keep popping up all over.

Although much has been written about benefits and drawbacks of working from home and about whether Marissa's strategy is good or bad for Yahoo, that's not really the crux of this debate. Much like the assassination of the Archduke Franz Ferdinand was just a pretext for World War I, but not the real reason for it, the outrage about Mrs. Mayer's decision stems from its potential effects on the society at large.

It's amazing how many people still get this wrong. For example, Rebecca Greenfield of The Atlantic Wire writes that we should chill out, because Marissa Mayer's memo is not about us:

But when Mayer declares an end to working from home, it does not in fact put an end to working from home elsewhere. Which is a relief because, if we've learned anything, some people really don't want to work in an office all the time.

It sounds reasonable on the face of it, but it's either downright disingenuous or just misguided optimism, depending on how generous you feel when you read it.

If you really think Marissa Mayer's decision is only about Yahoo, then you're living a sheltered life. Regardless of its position in the Silicon Valley's pecking order, Yahoo is still a large, prominent company and, as such, it can and does influence the society at large. Right off the bat, there were places where Mayer's move was taken as a sign that working from home should be banned.

Take, for example, this article from The Economic Times. Here's a choice quote:

"Flexitime is a utopian concept that is not going to help anyone," says K Ramkumar, Executive Director, ICICI Bank. "Whatever is not natural to the market and commerce, will not work. Customer is the king."

You might doubt the quality of the source, but it does not matter whether Mr. Ramkumar really said that or not. What matters is that there are numerous companies and managers who think like that and who must have been elated by the implicit validation they received from Mrs. Mayer's decision.

Here in Chile, the prevalent management style has been given a colorful name: "The Ranch Owner". What that means, in short, is that a lot of managers here expect their commands to be taken as divine law. For them, the topic of employees' happiness can be summed up with "I pay you to work. Whether you're happy or not is your problem."

What I'm driving at is that high-profile decisions in a company like Yahoo have an effect on much more than the company itself. The ripple effects of those decisions can affect a large part of the world.

To mitigate the impact of her decision, all Marissa Mayer had to do was make her telecommuting policy a bit more moderate: allow working from home on a case-to-case basis and, perhaps, choose a better wording that acknowledges telecommuting as an option that can enhance the quality of people's lives in general.

Instead, she's being defended and even praised for making a tough, controversial choice for the good of her company. The effects of her decision are being swept under the carpet of "fostering innovation", by people like Rebecca Cooper in her article for Washington Business Journal:

If she thinks bringing all her workers together in one place to innovate is the way to rebuild the brand, then I think we should let her try. I'm all for companies trusting their employees and giving them the ability to perform anytime and anywhere they can flourish. But I also think we shouldn't react so quickly to condemn this CEO for trying some old-fashioned togetherness to reboot Yahoo.

Even more chilling are the stories that praise her for "basing her decision on data". There's nothing wrong with consulting VPN logs and finding out that a lot of people are abusing their telecommuting privileges. Getting from there to outright banning telecommuting with no case-to-case considerations has nothing to do with data. Promoting the idea that you can make any decision infallible given enough data is downright irresponsible, especially when supported by flawed arguments such as this one:

Once, a Google designer quit the company in a huff because he was tired of how Mayer, in charge of how Google.com homepage looked, would choose design elements like color or font not based on taste, but raw data.

For every design variable, she looked at how users interacted with Google with one design — and then the other.

If the data showed users were using Google.com faster one way instead of the other, that particular design choice won out.

It's hard to argue that Mayer's process didn't work for Google. It was not the first search engine on the market, but it's just about the only one anybody uses now.

Conflating Google's success as a search engine with Mayer's decision making process about Google home page design is naive, but it takes a whole new level of self-delusion to use that as an argument to blow Mayer's analysis of VPN logs out of proportion and defend her decision as "data-driven".

Big companies nowadays parade their social responsibility initiatives and milk them for all they're worth. Yet when it comes to having a care about how your words might influence the lives of workers across the planet, many people seem to think it's not such a big deal.

After all, why should we care about the power of words? "Sticks and stones will break my bones, but words will never harm me." If that's what you subscribe to, then maybe you should reconsider.

The Great Programming Debate

noreply@blogger.com (Anonymous) — Mon, 04 Mar 2013 14:27:00 +0000

Over the last few years, I've noticed a pattern that keeps repeating in Internet forums and blogs:

Someone asserts that programming is or should be for everybody, usually misrepresenting it as a glamorous and easy job full of benefits and no drawbacks, like this.
Someone else posts an emphatic counterpoint, like this.
A third person tries to point out that a certain degree of "programming literacy" should be required and goes on to point out that everyone knows (or should know) how to read and write, like this.
The discussion devolves into polite bickering, because at this point most people are usually past their threshold for completely unbiased discussion and are invested in their own point of view.

I happen to have my own point of view, but it usually gets lost in the noise, since I invariably try to contribute it in a discussion during step 4. Looking back, I can see I've weighed in on both sides of the debate and I think it might be interesting to try to share a more balanced view and, perhaps, bring the people from the two extremes a bit closer.

Why Programming Should Be For Everybody

As I pointed out recently, in a similar discussion, I got into computers when I was 7 years old. Back then, "getting into computers", at least in my country, meant getting something like ZX Spectrum or Commodore and either playing games or learning how to code. Games were in English and a lot uglier than their arcade counterparts, so it should come as no surprise that most kids who had computers went on to learn how to make them "do tricks". That's the real allure of programming for a kid: you're typing in seemingly incomprehensible stuff and the machine does something. It's magic!

Not everyone had a computer, mainly because we were living in Communist Yugoslavia and it was not easy to import one, but nobody told us that computers are "not for everyone" and that only trained professionals can program. It was a slightly weird hobby to have, but it grew into some of the best years of my life: I grew up believing that the world would be transformed and that I would be part of it. Some days I felt like a combination of a wizard, secret agent and super-hero; other times I felt like a rebel straight out of Mentor's Hacker Manifesto.

At no point did I feel like a kid who's playing at being adult and should leave it to professionals.

You might think I'm attacking a straw man here, but this sort of elitism seems to be increasingly more prominent in discussions nowadays:

Sorry, but those things are technology and should be worked on by technologists because there's literally no on else who can work on them

That choice quote came from a discussion on Hacker News and it's the kind of extreme point of view that provokes people to react with very emphatic counterpoints.

Why Programming Isn't For Everybody

The problem with emphatic counterpoints is that they tend to go to the opposite, equally unhealthy extreme. If you've watched "What Most Schools Don't Teach", then you know that the intentions and goals behind the video are laudable. You probably also noticed that programming has been grossly misrepresented.

If you didn't know better, if you were the stereotypical impressionable youth, you might have walked away from that video believing that programming is easy if you know addition, subtraction and multiplication tables and that working as a programmer is a lot of fun, where you get free gourmet food, play video games and jam with a band inside an awesome office. You might also get an impression that you would be programming flying robots, self-driving cars and medical equipment.

As much as I agree that programming should be taught in schools -- and I'll get back to that later -- the fact is that programming is not easy and good programming is damn hard. To make things harder, good programming is a lot more important today, precisely because computers are so widespread. Bad code does have the power to ruin lives. I'm not talking about medical equipment or space shuttle systems here: a bad credit report won't kill you, but it might ruin your life for years; a badly programmed system might incorrectly match you as a terrorist.

When conveying a positive message in service of a good cause, you must take care not to go to extremes. Nobody tells kids that being a doctor is as easy as putting on a stethoscope and learning which side of the body holds the heart and which the liver. By simultaneously downplaying the difficulties of programming too much and painting a glamorous picture of it, you're setting kids up for disappointment:

We’ve all been raised on television to believe that one day we’d all be millionaires, and movie gods, and rock stars. But we won’t. And we’re slowly learning that fact. And we’re very, very pissed off.

-- Chuck Palahniuk, "Fight Club"

Incidentally, you're also bruising the egos of everyone who has worked their ass off for more than ten years trying to get closer to being a master programmer. While that's the most forgivable mistake, it still merits being mentioned, because you're alienating the very people you need the most.

Shades of Grey

In the end, it all boils down to recognizing that things aren't black and white. Putting the programming on a pedestal, unreachable by all but a select few, is as wrong as representing it as a glamorous job where anyone can be a rock star if they only try.

I agree that all schools should teach a certain level of programming, just as they teach literacy and a certain level of mathematics. In our society, computers are everywhere and everyone should be brought up to understand them and be reasonably proficient at manipulating them.

On the other hand, even though everyone is taught reading and writing, not everyone ends up being a writer. Even those who do are not necessarily successful. Likewise:

Knowing basic biology does not make you a doctor.
Learning how to play an instrument well enough to play it professionally requires constant practice and even then you might never get to be famous and hugely successful.
Getting into NBA requires more than being able to run the length of the basketball court.
Being able to prepare scrambled eggs or even come up with a recipe or two of your own doesn't make you Jamie Oliver.

Are these analogies exaggerated? Yes, but so are some messages we offer about programming. Let's face it, not everyone will have the aptitude to take make programming more than a hobby.

The important thing is that people of all ages need to be free to try it and we need to welcome them. However, it's equally important to make sure it's okay for them to fail. Our industry needs all the good hands it can get, but it's hard enough as it is without adding tons of bad, unhappy programmers who were promised the world and tricked into believing they could get it by wishing upon a star.

Another Step in The Longest Journey

noreply@blogger.com (Anonymous) — Fri, 22 Feb 2013 02:10:00 +0000

Over the last two weeks, I've been tweeting quite often about a certain project on Kickstarter. It's a game called "Dreamfall Chapters: The Longest Journey" and, if it doesn't ring a bell at all, I want to tell you why I'm so excited about it.

I've always loved video games, for the same reason I love -- and practically devour -- books: they tell me magical stories, take me to incredible places and allow me to do impossible things. The first game I ever played was "Chequered Flag" on ZX Spectrum 48 that my parents gave me when I was 7 years old. Since then, I've watched the history of video games unfold and it has been a fascinating journey.

Right now, the gaming industry is in a state that I find pretty disappointing: there's a bunch of big players that are mostly staying within the bounds of their franchises and churning out sequels following their tried, true and increasingly tired formulas. My prime example is Ubisoft. They won my heart with "Prince of Persia: The Sands of Time" and "Assassin's Creed". Then they broke it with "Assassin's Creed III". To be precise, they didn't break it all at once: they chipped away at it until it broke. It was a blow to me when they abandoned the beautiful "Prince of Persia" reboot in order to squeeze out "The Forgotten Sands" like another bowel movement, but what really eroded my respect for them was the way they kept stretching Desmond's story in the "Assassin's Creed" franchise until it felt like I stepped in a piece of bubblegum. Another example is Bethesda, although this one is much more positive, because they haven't (yet) pissed all over their players. The worst thing they've done is the progressive dumbing-down of "The Elder Scrolls".

Truly different games are rare and precious and if I had to single out a few, I would certainly start with "Shadow of the Colossus" and "Heavy Rain". Both throw away most of the conventions held dear by other games and come up with something unique. The former tells a pretty minimalistic story in a profoundly haunting way that leaves you in a pensive mood. The latter combines the best of reading books and playing video games: it's incredibly rich and immersive like a good book, but interactive and with a flexible story like a good game.

On the bright side, extreme innovation is not required to get a great game. Sometimes a brilliant execution of old gaming formulas will do the trick, such as Rockstar's "Red Dead Redemption" and "L.A. Noire". What makes these games exceptional is the way they explore profound themes through rich stories wrapped in engrossing gameplay.

This is why I'm so enthusiastic about "Dreamfall Chapters". For those of you who don't know the history behind it, it's the third and final part in the saga that started in 1999 with "The Longest Journey", a point-and-click adventure. The second part came out a whopping seven years later in the form of a third-person 3D adventure called "Dreamfall". Both games feature not only exquisitely rich and memorable world-building, but also excellent storytelling, with memorable characters and thought-provoking motifs, and beautiful graphics. And now, in 2013, Ragnar Tørnquist and his team at Red Thread Games are working on the final leg of a truly epic journey, wholly deserving of being called the longest: "Dreamfall Chapters".

Now, let's step back a bit here and focus on one important fact: how much passion, patience and balls does it take to stick to a dream over the course of 14 years? For that alone, I would have been inclined to back "Dreamfall Chapters", even if I hadn't played or heard of its prequels.

That, however, is not the primary reason why I'm trying to do all I can to promote "Dreamfall Chapters". The real reason is because I know Ragnar and his team are going to produce another masterpiece that will not only bring enjoyment to countless players across the world, but will also bring a breath of fresh air to the gaming industry.

Here's why I trust the Red Thread Games team: I first played "Dreamfall", without even having heard of "The Longest Journey". I played the whole game and enjoyed it immensely, without having any idea that it's a sequel. I never felt lost. There was never a "something doesn't end up" moment or "am I missing something here" feeling. And yet, when I discovered "The Longest Journey" and played it, not only it fit perfectly, but it transformed "Dreamfall" for me. So far, I've only had one other experience like that: when reading Discworld books by Sir Terry Pratchett. If you've read Pratchett's work, then you know how high a praise that is.

By all means, don't take me wrong: I don't claim these two games were perfect. "The Longest Journey", for example, had one of those "dude, wtf?" puzzles that old-style adventure games were so criticized for; just look up "the longest journey duck puzzle" on Google. "Dreamfall" had a totally superfluous combat system that makes a game of "Pong" feel like "Ninja Gaiden" in comparison. Yet, despite these imperfections, these games were -- and remain -- some of the most enjoyable adventure games out there.

So far I've covered why I trust the team, but I haven't really given much details about why I believe this to be so important. So it's another adventure game, albeit an excellent one, so what?

I'm very passionate about games. Apart from enjoying them as a player, I'm also one of those few adult programmers who hasn't outgrown the dream of making them. For me, it's important not only what games are made, but who makes them and how.

Ragnar Tørnquist left Funcom and founded Red Thread Games in order to make "Dreamfall Chapters". Don't get me wrong, I have nothing against Funcom at all, but the fact remains that a well-known game designer and one of the iconic figures of game development industry left an established, stable studio and went indie to make this game. And it's not any game, it's the long-anticipated finale to a saga that has, despite any imperfections we might find in it, earned its place in the history of video games.

I want to see it succeed and I want it to be a huge success. Apart from being a work of art, it will also challenge the status quo by showing that it really is possible to make a masterpiece without clinging to the old publisher-developer structure. And, as a bonus, it will contribute to Linux and Mac gaming scene and bring back some of the glory to PC gaming, wresting a bit of power from the tightly controlled walled gardens of the console world. All in all, it should be an important step in a journey even longer than "The Longest Journey": the journey through the history of gaming.

If you're still with me after all these words, then I hope I've convinced you to at least go to the Kickstarter page for "Dreamfall Chapters" and look at what you find there. And after that, if you feel moved to back the project, then so much the better.

Fix my car!

noreply@blogger.com (Anonymous) — Sun, 18 Nov 2012 02:14:00 +0000

Late Friday afternoon stretched slowly towards the evening. The A/C inside Mike’s auto repair workshop had been fighting a losing battle with the midsummer sun all day. To anyone coming in, the noise and the bustle were disorienting, and the smell of grease, oil and exhaust fumes was nearly overpowering. To Mike himself, it sounded like a hum of a busy day and smelled like the challenge of honest work.

Elbow deep into the hood of a ‘99 Jetta, whistling absentmindedly (and off key), Mike was concentrating on a tricky bit of engine repair, when a tap on his shoulder startled him.

“Excuse me,” came a tentative, almost plaintive voice from behind.

“Be with you in a jiffy,” said Mike, as he began to extricate himself. In just a few seconds, he was free and wiping his hands and arms on a rag as he studied the neatly dressed, bespectacled young man in front of him. “Hi, I’m Mike. How can I help you?”

“Yeah, hi. My girlfriend’s car is broken and I need someone to fix it.”

“Well, then you’ve certainly come to the right place! Bring it in and let’s have a look at it.”

“Uhm, I can’t. It’s not here. It’s parked in my driveway.”

“Ohh-kay.” It took Mike all the self-control he learned in more than a decade of running his shop to keep his face politely impassive. ”What make and model is it?”

“It’s... um, I’m not sure. It’s green.”

“Green. Right.” Silently counting to ten in the back of his mind, Mike tried a different tack. “Right. And what’s wrong with it?”

“I don’t know,” the young man in front of him huffed exasperatedly, “I was hoping you could tell me.”

“What I’m asking,” said Mike, surprised to find he’s not gritting his teeth, “is why you say the car is broken? What happened?”

“Oh. That. Sorry. It doesn’t move.”

“Uh huh... Anything, um, more specific?”

“Nope. Don’t know the details.”

“Well then,” shrugged Mike apologetically. “I’m afraid there’s not much I can do to help.”

“Okay, fine. Hang on a second.” With that, the young man whipped out a cell phone and punched a number into it. “Hi, honey. Yeah, I’m at the auto repair shop. Look, I need to know what’s wrong with the car... Yeah, I know, that’s what I said too, but they need more details to do their job... I know, right? … Uh huh. Uh huh... Okay, great. Love you. Buh-bye!”

“Right, so, the problem is that it doesn’t move when she puts it in the gear. She steps on the pedal and the car just sits there and roars. The wheels won’t move.”

“Okay,” said Mike, “now we’re getting somewhere. It sounds like a transmission problem.”

“Transmission? What do you mean? What’s it transmitting and to whom?”

“No, no, it’s the mechanism in your car that allows the wheels to move when you put it into gear. That mechanism seems to be the problem.”

“Right, that’s what I just told you. Now, when will you fix it and how much will it cost?”

“Well, when you bring it in, I can look into it and diagnose the exact problem.”

“Didn’t you just say that this transmission thingamajig is the problem?”

“Yes, but the transmission mechanism is a complex system with lots of components. I need to look at it to find out exactly what is broken and how to fix it.”

A suspicious look settled on the young man’s face. “I don’t know,” he said, arms crossed, “It seems to me like you should be able to tell me how to replace this transmission system.”

Mike tried to find something to reply to that and failed. He opened his mouth a few times, but before he could come up with a way to deal with this Kafkaesque situation, the young man threw his hands up in the air and said, “Fine! Alright. I’ll bring it in tomorrow. Anything else?”

“Not that I can think of,” said Mike cautiously.

“Good. See you tomorrow, then.”

“Unless I’m having a nightmare,” murmured Mike into his beard, as he watched the young man walk out in a huff. Slowly shaking his head, not sure whether what just happened had been real, Mike turned back to the Jetta. “Thank heavens my customers usually aren’t like that...”

So what do you think of Mike’s story? Did it sound absurd, almost surreal? Granted, there are all kinds of people out there, so it’s not impossible to stumble into someone like Mike’s strange client, but it sure is uncommon.

What if I told you the young man in the story worked in a car factory? Wouldn’t that make the story outlandish?

And yet that’s precisely what happens quite often in software development. Many smart people have already spent lots and lots of words on how to report bugs, how to ask for help, how to follow up and how to ask questions in general. I don’t have much to add to all that, but the next time you complain that some piece of software isn’t working without giving any relevant details, remember this story. Doubly so if you actually work in something related to software development.

My Playthrough On Your Fridge Door

noreply@blogger.com (Anonymous) — Fri, 23 Mar 2012 03:50:00 +0000

Two days ago, I decided to take a brief break at work. Since I was still gloomy from having finished Mass Effect 3 that weekend, I went surfing the Web to see what's new on that front and stumbled upon a couple of articles that inspired me to write a new post on my blog. Only two days have passed since, but it seems much longer.

To be honest, I didn't expect to succeed at generating any kind of discussion about this. Most people, perhaps predictably, responded with a knee-jerk reaction along the lines of "Yes, you're just audience, so shut up!" Finding that there were people who agreed with me was quite gratifying. Finding that there were people who disagreed and wanted to discuss it, however, took me completely -- and pleasantly -- by surprise.

I don't blog much. To quote Snoopy, "there's no sense in doing a lot of barking if you don't really have anything to say" and I usually don't have much to say that other people haven't already said more eloquently or elegantly. To find myself blogging for the second time in one week is a new experience. I'd like to thank all the people who took their time to comment and to converse with me in various forums: thanks for inspiring me to write more.

Enough of meta-blogging. Let's get to the meat of the discussion: over the past two days, I've seen a few valid and interesting counter-arguments to my claim that players aren't merely an audience any more. I'd like to address those arguments here. (By the way, if you haven't finished Mass Effect 3 and don't want spoilers, be careful when following links from here.)

Let's start with the one that's easiest to refute and take it from there.

Playing Games is Not Art

Hobbes: Look, it's just a game.
Calvin: I know! You should see me when I lose in real life!

-- "Calvin and Hobbes" by Bill Waterson

No, it's not. I agree completely. Maybe some day, playing certain kinds of games will become an art form, but right now, playing games is definitely not art.

Thing is, I never claimed it was. Don't get me wrong, I would love to see the day when there are games that allow players to use them as a medium for their artistic expression. But for now, what I claim is that playing games -- at least some games -- is an act of creativity. I also claim that players, by playing the game, complete the creative process the game developers initiated.

Although not as strong as "playing games is art", this claim still sounds weird. The reasoning that rejects this claim tends to fall into two categories. The first centers on:

Finite, Limited Universe of Expression

My arena of literal expression offered

four avenues to the topic of elimination,

two references to human anatomy,

one request for divine imprecation,

one standard description of or request for coitus,

and a coital variation which was no longer an option for me

since my mother was deceased.

All in all, it was enough.

-- Martin Silenus, from "Hyperion" by Dan Simmons

Arguments in this category reject players' participation in the creative process because said players are restricted to a finite and rather limited set of options, rigidly established in computer code by game developers. As Michael Rohde neatly put it:

[...] you are playing code, you are not helping write the story line, you do not have the ability to write unique aspects of the game. You are navigating a trail with finite endings.

The problem with this argument is that it's ultimately quantitative in nature. Just because English language has a finite number of ways of putting its words together in non-gibberish ways, doesn't meant that writing a sonnet -- a rigidly defined and limited form in itself -- is not a creative act. We have Shakespeare to back that one up.

To put it differently, if you're given a small set of Lego pieces, you can still put them together in a unique, creative way. In case of Mass Effect games, the sheer combinatory explosion of possible choices to take in the game is at least comparable to a box of Legos. Even though those choices converge on an extremely small set of final outcomes, the act of making those choices deliberately is no less creative.

That's the second problem with this argument: by focusing only on the creative aspect of the game's fixed assets, such as the story or the cinematics, it misses the fact that these assets themselves serve as a vehicle for players' creative expression. What each player in Mass Effect 3 creates is the character of Shepard and his or her arc. Aleksander Adamkiewicz expressed this nicely in his comment on a related Gamasutra article:

[...] I always thought Shepard was an empty vessel for the player to fill with their interpretation. [...] I never found that Shep undergoes any character development in the narrative of the game, yes he underwent a development -in my head- but not in the game itself.

That is precisely the allure of Mass Effect series, the characteristic that sets it apart from so many other adventures and RPGs. I posit that almost all players go through this "Shepard-development in their heads". Some do it more consciously, more deliberately than others, but everyone who's playing the game for more than "shooting stuff" does it.

Still, even this might not be enough to accept the players as creative agents. There's still one more category of counter-arguments to tackle:

The Question of Intent

The most exciting phrase to hear in science,

the one that heralds new discoveries,

is not ‘Eureka!’, but ‘That’s funny …’

-- Isaac Asimov

My son's fifth birthday was in February, when most of his friends are out of town on vacation, so we threw a party for them last Saturday. This year, the main attraction was a magician. At one point, he invited my son on stage to "grant him magical powers", so he can "assist" the magician.

Asking for a volunteer in the audience and inviting him or her to participate is a standard part of a good magician's repertoire. However, this act of participation does not turn the audience member into anything more than audience. Their participation does not promote them to the status of the magician.

Why, then, do I argue that the act of playing a videogame can promote a player into a creator?

The crucial difference is in the intent. The magician doesn't intend to truly empower you, while a good game designer strives to do precisely that. Game developers have been acknowledging the players' desire for creative expression for years. One small example is Burnout Revenge, a game whose Xbox 360 version turned six years this month. A notable addition in the Xbox 360 edition of Burnout Revenge is the Burnout Clips feature, which allows you to create video clips of your offline races and share them with other players.

A much more important example is Little Big Planet, along with its sequel. The sequel, in particular, is the best example of a game whose sole reason for existence is to allow players to create their own content. Its single player campaign, unlike the one from the first game, is primarily a showcase for the new game mechanics you can use in your own levels.

And let's not forget other, much older "build your own stuff" games, such as Sim City. Sid Meier took this idea and turned it into an art form, if you'll pardon my choice of words, with his Civilization games.

Okay, I'm digressing and I'm sure you get the point. Let's get back to the game that sparked this whole discussion.

Was there intent to empower in Mass Effect? I would say that if there wasn't, BioWare is the luckiest company on the Earth. That kind of repeated serendipity is difficult to believe in.

Yes, I dare say that Mass Effect was deliberately designed to empower the players to build their own Shepard character in their heads.

Is this an act of creativity? I believe it is. If you see a reason why it shouldn't be considered as such, feel free to drop me a comment.

More importantly, has this creativity been deliberately encouraged by BioWare? I don't see any room for doubt here.

Incidentally, this is precisely why so many players are so upset about the ending. Like I already said in my previous post, the ending completely invalidates the players' choices. Yes, we're all used to our choices finally converging on a small set of endings, as in the previous two installments. However, for various reasons (which I won't reveal in order to avoid spoilers) the ending(s) in Mass Effect 3 are different: players' choices, instead of converging, are simply discarded and ignored.

Does Any of This Matter At All?

Bun-Bun: This is pretty crappy paraphrasing.

Riff: You should see what I have to work with! It's in crayon!

-- "Sluggy Freelance" by Pete Abrams

People who have children are very well acquainted with one amusing fact about creativity: the vast majority of little kids completely suck at drawing. That's perfectly normal for their age, though, so we ignore that fact and we praise them for their creativity and stick their doodles on the fridge door or on our cubicle wall. It's not because they're great works of art. It's because they matter to us, because our own children drew them.

Similarly, most of players' creations in games that encourage creativity are, well, the digital equivalent of a five-year-old's "crayon art". It's up to game developers to define their stance towards our creations.

Am I wrong to believe that they should try to be like proud parents, indulgent and encouraging? Only time will tell. In the mean time, let me know what you think.

Are Players Just Audience?

noreply@blogger.com (Anonymous) — Tue, 20 Mar 2012 18:02:00 +0000

Part 1: Sisyphus

Over the last weekend, I finished my first playthrough of Mass Effect 3. For the moment, it looks like it will be the only one. My immediate reaction was that of vague disappointment. I felt dissatisfied, but I couldn't put my finger on why. It took me a bit of introspection to come to the same conclusion I later saw in countless blog posts, tweets and articles: the ending completely invalidated all the effort I had put into the game.

All those hours of chosing my answers carefully, pondering what I would do if I had to face a situation like the one Shepard was facing, wondering whether I was being consistent and, in general, carefully crafting a character and learning more about myself in the process -- all that effort was for nothing. The game gave me three options, all of them cataclysmic and none of them even remotely satisfying. Under my guidance, Shepard had achieved what no sentient life form had ever achieved before, but it didn't matter, the choices were still the same. One of those choices went completely against every major decision I have ever made, since Mass Effect 1, but that didn't matter, the choices were still the same. And after I took one that looked the least unpleasant, I was treated to a cinematic that opened new questions while providing no answers whatsoever and left with no closure and a bad taste in my mouth.

"Charlie Chaplin entered a Charlie Chaplin look-alike
contest in Monte Carlo and came in third. Now that's a story.
This... is something else."

-- Mr. Goodkat

Later, I sat down to read about other people's reactions. A small part of me still wishes I hadn't done that. Up until then, my misery was based only on my own experience. And then I read about other endings. Nominally, there are sixteen of them. In reality, it's 90% the same ending, with variations in small details. In other words, it wasn't merely something I had done in this playthrough. No matter what I did, no matter how many times I played the game again and again, I would never get a significantly different ending.

They say that misery loves company and it turned out to be true in the end. While finding out about other endings made things worse, I was strangely comforted by the fact that I wasn't alone and that other fans were speaking up about this and voicing their -- or rather, our -- outrage. Further reading brought small glimmers of hope: official Bioware responses could be interpreted in a way that left open the possibility of changing the ending.

I wasn't going to write about any of this. Others have done that more eloquently and I've never acquired the taste for shouting "me too" in the crowd. What changed my mind was Ken Levine's reaction. And then Ben Dutka over at PSX Extreme took it to another level. And suddenly I couldn't stay silent anymore.

This argument wasn't unexpected. On the contrary, it's a fairly standard line of defense and usually it's completely valid in certain aspects. Usually, but not anymore. Videogames have changed that and people like Ken Levine and Ben Dutka are either not aware of it or don't want to accept it.

Before I get to what has changed, however, I'd like to go over the things that haven't. It's not just my flair for dramatic. I want to clear the air first. As a geek, I have a natural distaste for bullshit in all its forms and I want to make certain things clear to both my readers and the people out there who are making their righteous statements about artistic integrity.

Ben Dutka is right, up to a point: we don't have the right to walk up to an artist and demand from them to change their work. Or rather, we don't have the right to have that work changed just because we demanded it; whether we demand it or not is merely a question of etiquette, not of rights. We have not been granted any intellectual ownership rights by purchasing the work of art or anything of the kind. We've been granted the right to experience that work of art. Whether we enjoy it or not is immaterial.

The problem with that attitude is that people use it to hide from the rest of the ugly truth: art has never been able to stand completely on its own. It's not just an independent fact, an axiom. It's the product of a process in which there are several crucial roles. If we don't acknowledge this, we're simply not being honest.

Part 2: Apollo

A work of art needs an Artist to make it and an Audience to appreciate it. Otherwise, it's simply a tree that fell in the forest with nobody to see or hear its fall. The cold, hard, ugly truth is that the Artist needs a Patron. Either that or a Day Job, but we'll get to that in a moment. The point is that the Artist, like any other human being, needs to earn a living.

Patrons used to be reasonably wealthy people who would commission Artists for specific projects, e.g. "Paint my wife, dude, I need something to hang in my new home!" Over the time, the business model shifted. The patronage became crowdsourced and the Middlemen stepped on the stage. Nowadays, the Artist will typically go to a bunch of Middlemen for help with any and all of the following: getting the means to create a work of art, gathering an Audience for a work of art, putting the Artist together with other Artists to collaborate on a work of art and, ultimately, getting paid for a work of art.

The rise of Middlemen wasn't the only consequence of patronage being crowdsourced. Art and entertainment started overlapping a lot more than before. The Audience demands what Kurt Cobain sang: "Here we are now, entertain us!" You want the patronage? Deliver!

Like it or not, but the fact remains that the most famous modern Art is inextricably entangled with Entertainment and Business. Sure, you can still adamantly refuse to compromise your artistic integrity. If you're very, very good, you might be able to get away with it. If not, you still have a perfectly valid option of practicing your art as a hobby. The stereotypical image of "Starving Artist" comes from people who refuse to accept this reality and try to be professional artists without an impressive track record and with rigid artistic integrity.

Ben Dutka can lament this reality as much as he wants. He can warn us that it leads to Art degenerating into "some hideous, mutated, mass-generated assembly of likely sophomoric compromises". Guess what? We've already arrived there. That place is called Hollywood. The last Hollywood movie I myself have seen that stuck out as memorable work of art was "Inception" and even that was watered-down art.

"Tell it to the one legged man so he can bump it off down the road."

-- Sloe

Do I want videogames to degenerate into Hollywood-like decadence? Hell, no! Things are bad as they are: just look at Assassin's Creed and how it's getting stretched beyond recognition, worse than the "Wheel of Time" used to be before Brandon Sanderson took over. No, thanks. I'm all for artistic integrity. But let's not pretend that things are not the way they are.

And, by all means, let's not allow the defense of artistic integrity to become the reason for the Middlemen to further crush it and suppress it. Get it in your heads, Artists, those Middlemen have the power to crush your integrity and they will not hesitate to use it, because that's how they make their living.

Mass Effect 3 has the potential to become the textbook argument for publishers to impose their will on artists: "Yeah, we let you guys do your artsy-fartsy stuff in Mass Effect 3 and look how that turned out. Look at this sales chart. Look at it, damn you!"

I'm sorry you're sad, Ken Levine, but you're also wrong. Leonardo da Vinci was told to paint the wife of Francesco del Giocondo. He could have decided to paint the lake behind her, instead, but he didn't. He did the work he was hired to do. Was he "disappointed in the emotional feeling" he got because he "didn't really create it"? I guess we can't find out because he's not around to give us an interview, but I look at La Gioconda's smile and I doubt it.

Part 3: Orpheus

None of that, however, was my main point. All of the preceding discussion applies to the traditional artistic media. The videogames have changed the equation and that's really the bone I want to pick with Ben Dutka.

I think the fact that video games are interactive, and the fact that gamers have more control than ever (customization, user-creativity, etc.) has confused things but the fact remains, a story is a story. It's art, regardless of the medium.

I take issue with that claim. It's the same logic that RIAA and MPAA use in their claims and in their fight for SOPA and PIPA: "Nothing has changed." Everything has changed, Ben, and it's time to face it.

You cannot be a game designer without learning one important lesson: you are not telling a story, you are helping your players create their own stories. One of my most favorite books on my non-fiction bookshelf is "Game Architecture and Design", by Andrew Rollings and David Morris. I still remember the sense of wonder and realization when I read the following words:

Even puzzle games like Tetris in one sense tell a story and are thus "dramatic" to an extent.

Reading those words, I flashed back on all the times my buddy and I would get together and excitedly discuss the game of Descent II we just played over the modem; I remembered the Elven Runes website where people would post their logs after an exciting session of MUME. The image these words painted in my mind was surprisingly vivid: a bunch of people gathered around a fire at night, swapping stories. For a guy who has never had an actual campfire experience, that image was most alluring and the realization that this experience is, in a sense, what videogames bring to countless players around the world was... mindblowing, to put it mildly.

Just what role does your story play in the game you create? That's not a new debate. Just ask Ernest Adams what he thinks about it. Still not convinced? Digital Worlds has a post about the whole debate.

Here's the bottom line: we, the players, participate in the creative process and our participation is crucial because it's the final step. The players complete the creative process.

"I live on both sides of the fence. My grass is always green."

-- The Rabbi

And that, Ben, is what gives us the right that no other audience had before: the right to demand that our role in the storytelling and creative process be honored and respected.

Do I have a legal right to demand a satisfying ending for Mass Effect 3? I don't think so, although at least one person disagrees with me.

Do I have a moral right? I believe I do. I paid BioWare and EA what I owed them. But whether they like it or not, they owe me, too.

Yes, Ben, I want to set a precedent, no matter how dangerous you think it is. I want BioWare to recognize the fact that their audience is not just an audience anymore: we're minor artists ourselves. We're storytellers. We watch the tree fall in the forest and then we go and build our houses from it.

To people like Casey Hudson and Paul Barnett, these creations might not be as important as their own artistic integrity. But the least they and the rest of BioWare could do is not tear them down.

So what'll it be, guys? What kind of designers are you?

If you liked this, check out the follow-up: My Playthrough on Your Fridge Door.

Old School or Just Old?

noreply@blogger.com (Anonymous) — Tue, 08 Nov 2011 01:58:00 +0000

I’ve been looking for a diplomatic way to start this rant. After several false starts and a couple of glasses of wine, I realized that beating around the bush wouldn’t do it. Instead, I'll dispense with the bullshit and jump straight into what’s bothering me:

I’m tired of being immersed in a culture of mediocrity.

There, I’ve said it. Now I have to spend the rest of the post explaining what the hell I meant by that. Don’t expect me to beat around the bush, either. If you disagree with what I say, fine, but I’m not going to sugar-coat what I think, because too much sugar-coating is part of the problem.

I want to make one thing clear right from the start: I have a good job. There’s a lot to love about it. For one thing - the most important one, really - the work I do is interesting, challenging, ambitious and with plenty of opportunities to learn new things. It’s the kind of work that I often wished I could do and the fact that my wish came true is something to be happy about.

Plus, the compensation is good enough to allow me and my family to live comfortably. To top it off, my boss is a nice dude, trusts me to do good work, helps me out when I need it and doesn’t make a fuss when I need to telecommute.

Why, then, do I describe all that as a "good job", instead of, say, "dream job"?

I guess I could blame it all on my parents. They helped me discover computers and programming when I was six and it has been my passion since then. I’m almost thirty-three now and it has been a wonderful journey so far. Most of the stuff I can do, I discovered by trying to do it and looking for information on how it can be done. Not that I mean to imply I can do or learn anything. On the contrary, there have been things I tried to do or learn and gave up. The important thing, to me, is the fact that I acknowledged the fact that I gave up: for whatever reason, I chose not to expend the necessary effort.

On that journey of discovery, I’ve had the privilege to meet quite a few brilliant people. The first time was in 1993. Having graduated as top of my class in the ground school, I sailed into my high school like Titanic and was promptly sunk by the realization that almost everyone was more brilliant than I was. It was one of the best times in my life.

Another was when I went to work for Synopsys. It had taken me a bit more than seven years and almost as many jobs to find them. Once again, I was surrounded by brilliant people, all sharing their knowledge and pooling their skills to overcome nearly impossible challenges and improve themselves and the software they were developing.

Like I said, I can blame it all on my parents. My dad used to say, "Don’t go to the army if you don’t intend to be a general." By that he didn’t mean I should be unreasonably ambitious; rather, he meant I should always do my best and try to get better. The idea is to aim as high as you can, work hard and see where that gets you.

That is why I get so pissed off by people who refuse to do their job without someone holding their hand. That is why I can’t fathom how people think they can change the code without understanding what it does or what they’re doing. That is why I can’t stand people who don’t even know the basic stuff anyone with their job description is required to know, but are quite happy to choose their own solution even after you patiently explain why they should do it differently.

The worst thing, however, is that you’re not allowed to say "No, I won’t have a half-hour NetMeeting session with you. You didn’t even try to understand what it is you’re doing. You just blindly tried to apply a completely arbitrary set of changes and when that didn’t work, you gave up and decided you need someone to do it with you. And no, I won’t do it for you, either."

Don’t get me wrong, I’m not ranting about the lack of knowledge. I’ve yet to see someone who was magically born knowing how to write code. I have no problem helping people who need a hand, either. What I’m railing against is the bad attitude, the kind you get in people who don’t even try. You can spot it right away, because they don’t even ask you questions. There’s a vast difference between:

I tried doing X, but it didn’t work. Then I looked at Y and I think it does Z, but I’m not sure why P and Q aren’t there. Am I supposed to do A and B before X?

and:

I tried doing X. It didn’t work and I don’t know why. Can we have a NetMeeting session so you can help me out?

To make things worse, managers often nurture that kind of attitude. For example, suppose you’re writing some code that has to do X and there’s a robust, mature open source library Y that helps you do X without reinventing the wheel. If you decide to use Y, then you’ve shackled yourself with maintenance of that bit of code forever. Whenever someone gets the task of doing something with that code, all they have to do is complain that they don’t know how to use Y and the task will most likely get reassigned to you. It doesn’t matter that you didn’t know how to use Y, either, before you decided to use it to do X. Apparently, you’re the only one who can learn how to use an open source library whose documentation is freely available out there.

That’s what I was referring to by "culture of mediocrity". It’s not just the belief that you don’t even have to try to learn something or expend some additional effort, it’s the belief you shouldn’t have to try. It’s the belief that being given a job is the proof that you know enough and if you don’t, it’s the fault of whoever gave you the job. It’s the belief that nobody has the right to expect you to go an inch beyond that. In extreme cases, it’s the belief that going beyond is only going to get you saddled with more work, so you’re a fool to do it.

By all means, don’t take this as a criticism of my workplace, specifically. It’s not endemic to the company I work for. Rather, it’s widespread among the BigCo type of companies. I’m not sure why that happens and why there are so few non-startup companies, like Google or Synopsys, that don’t allow that to happen on a grand scale. Frankly, I’m not in the mood to speculate about it, either.

Things aren’t likely to change, no matter what I say here. For my part, I refuse to adapt to that situation. I’ll take pleasure in working with the few brilliant people scattered around in the sea of mediocrity that surrounds us. I’ll grumble and groan about the rest, until the work stops being interesting, the pay falls below the comfort threshold or the nice boss gets replaced by someone not so nice; in short, until it's time to move on.

I like to think of myself as "old school", since I’ve gone from writing spaghetti code in ZX Spectrum’s BASIC and even worse Z80A assembler code to writing my own DSLs and compiling them to JVM. But maybe that’s not it. After all, I can often name the song playing in an elevator or a mall. I remember when MTV actually had music videos playing all the time. One of my favorite bands broke up after thirty years of playing together. I used to have a BBS that was a point in the FidoNet. Maybe I’m not "old school", but simply old.

Whatever. If you can’t be bothered to even try to learn new things, then get off my lawn.

Scrum and Politics

noreply@blogger.com (Anonymous) — Thu, 12 Nov 2009 12:37:00 +0000

Here's why so many people hate the Agile: consultants. Yes, many of them are nice people who truly believe in what they're teaching. Unfortunately, this doesn't change the fact that, at the end of the day, they get to walk away and you're stuck with whatever they did or failed to do.

Think about the process consultants for a moment: their job is to come to someone else's workplace and make them change the way they do things. They don't come uninvited, though; you have to invite them in, just like vampires. Once they're inside, they have to make things work the new way. Invariably, they will encounter resistance from people who are comfortable doing things the old way. They have to know how to filter out legitimate idiosyncrasies of a particular workplace from loads of bullshit coming their way. In other words, they have to be well-versed in workplace politics.

Above all, though, a process consultant has to demonstrate success, because that's what he's selling. The project has to be successful or the consultant's company loses face and money. And that's the bottom line: the project will be perceived as successful, no matter what. That's also the top reason why a consultant needs to be a good "office politician", because he has to turn casualties into "collateral damage".

Don't get me wrong. Like I said, a lot of those people are essentially nice; they don't do this stuff out of malice, but out of belief that they're helping. What I'm trying to explain here is that the road to hell really is paved with good intentions.

Why am I singling out Agile, though? Mainly because of the Scrum. When it comes to project management, Agile consultants will always reach for Scrum and that's the source of more than half of the trouble they bring. I hereby postulate that Scrum is especially vulnerable to office politics.

The Secret of Scrum

“If you can't get rid of the skeleton in your closet,
you'd best teach it to dance.”
George Bernard Shaw

A typical sales pitch for Scrum is based on the comparison with the "traditional waterfall" process. Don't let the consultants fool you: incremental and iterative development is not unique to Scrum. Neither is the idea that you should ship often. What really sets Scrum apart is a very simple idea: free the developers.

The whole "chicken and pig" philosophy, the existence of a Scrum Master who focuses on "removing impediments", the emphasis on self-organizing teams -- all of that aims to clear all the typical obstacles that a team faces when trying to develop a product. The theory is that, once these obstacles are out of the way, the team will be free to develop a great product. That's why they have all these people at their disposal, right? The team focuses on building great stuff and everyone else focuses on giving the team what they need. And then they all live happily ever after...

The most fascinating thing about it is that it really can and does work. I've seen a few teams that pulled it off. I've also seen a few that could have pulled it off, but the management wasn't inclined to let them try. The one thing that the consultants won't tell you or your bosses, however, is that not all teams can pull it off. That's Scrum's dirty, dark little secret. If you have a process that requires a self-organizing team, then it should be obvious that your team needs to be capable of organizing itself. It's surprising to see how many people neglect this issue.

Growing Up

“A person's maturity consists in having found again
the seriousness one had as a child, at play”
Friedrich Nietzsche

Let me spell it out clearly: self organization is a very difficult skill to obtain and maintain. The typical response to this concern is that maturity can be achieved in time and that this "only" implies a ramp-up period. While I agree with that sentiment, this is usually the point where consultants screw things up: typically the ramp-up period is one sprint or so and there's not enough support for the team. You can't just dump it on a team and expect it to "ramp up" to it. Sure, every good team wants to be free, but that doesn't mean they know how to be free. It's an ironic, but not unusual situation to obtain the freedom you've always wanted, only to realize that you don't know what to do with it.

Maturity, both for teams and their members, means many things. It's not just about adapting to the fact that you won't have a manager telling you what to do. It's also about having enough context. It's amazing how little context a typical software developer needs, as a minimum, in order to do his or her bit and live happily oblivious of the bigger picture. Many developers out there know a lot about the data model of the underlying systems or about this application server or that legacy code, but don't really understand what the products really do and what makes them successful. If you put such people on an important project and tell them to organize themselves, they'll be like fish out of water. They know how to attack technical challenges, but they don't know how to make the project succeed. This issue is also easily addressed, at least in theory: ensure that the Product Owner guides the team. In practice, however, the person who should be the Product Owner is often too busy to participate properly. In such cases, you either have an absentee Product Owner or a surrogate whose contributions are of dubious quality.

Control Chaos, But Avoid Anarchy

“It is best to do things systematically, since we are only humans,
and disorder is our worst enemy”
Hesiod

Yet another critical mistake some consultants will make is to take the idea of cross functional teams to an extreme. Personally, I think this mistake might be inspired by the name "scrum", which comes from rugby. The whole approach was initially inspired by a paper published by Hirotaka Takeuchi and Ikujiro Nonaka, called "The New New Product Development Game"[pdf]. In the paper, the waterfall approach is compared to a relay race and the new approach is compared to rugby, "where a team tries to go the distance as a unit, passing the ball back and forth." Other people later named it Scrum to emphasize the idea that, just like in rugby scrum, all team members must "push" together to overcome the challenges.

Ironically, people who put too much emphasis on this aspect tend to forget the bit about "passing the ball back and forth". Team members all have roles and they were put in those roles for a reason. Working together in a cross-functional team means that the team is composed of different functions that cooperate smoothly, not that every team member should be called to perform any function at any moment, even if they are capable of doing so. There's a reason you hired someone to be a QA analyst in your company. It doesn't matter if they happen to have the skills to do some other job too; unless they've been doing that other job -- in your company -- long enough to feel comfortable in it, they can't do that other job on as well as a person whose job it really is. On top of that, the money you're paying someone must have at least something to do with the job they're doing and with their contractual responsibilities. It's bad form to piss all over that in the name of "cross-functional teamwork".

Wagging The Dog

“Those who say religion has nothing to do with politics
do not know what religion is.”
Mahatma Gandhi

What does all this have to do with my initial claim that Scrum is especially vulnerable to office politics? Think about it: all of these issues can be, and often are, highly political. Saying that your team lacks maturity can be seen as an admission of failure. Alternately, it can single you out as the "negative thinker" or "that guy who opposes the change". And let's not forget that we're talking about a group of people here and that even if some of them are ready to admit the lack of maturity, others might take offense at this. All these are great incentives to sweep the problem under the carpet.

Maturity isn't the only political issue here, either. Take the Product Owner, for example. If the only person who knows enough to be the Product Owner is too high in the food chain, this means that not only he or she won't be able to participate as required, it also guarantees that nobody will want to ask them for guidance as often as necessary.

And heaven help you if you run into a consultant who's a "cross-functional extremist". Since Agile is all about moving away from "heavyweight" model, if you dare to pipe up about this issue you'll be accused of "insisting on ceremony" and "placing too much importance on titles". Do you detect the odor of politics yet?

Make It Work

“Organizing is what you do before you do something,
so that when you do it, it's not all mixed up.”
A. A. Milne

Okay, so Scrum can be thoroughly screwed up by office politics. So what else is new? It's easy to point out the problem, let's hear some solutions. As always, there's no silver bullet. There are, however, ways to avoid some common pitfalls.

First and foremost: help the team reach the necessary maturity. This is not a trivial task and it should not be underestimated. Remember all that time you're supposed to save by having a lightweight process with minimal overhead? Well, that doesn't come for free. Initially, you might have to invest a hell of a lot of time and effort into getting the team up to the point where it can truly be self-organized. Make sure that the right people are in charge of this effort. The person doing this should ideally be a project manager, hopefully in the Scrum Master role. Whatever you do, don't just dump this responsibility into the lap of whoever seems to be convenient. For example, your architects might seem like the good choice of being "team leads" who can take care of this, but you should remember that there's a reason you hired them as architects: they'll be busy making sure everything fits in the "big picture" on the technical side of things; yes, this does include guiding other team members, but not organizing and managing them.

Second, don't mistake teamwork and cross-functional approach for anarchy. The team needs to work in an organized way, which means that roles are there for a reason. Self-organization isn't a democracy, either. Everyone should be free to contribute, but if the key decisions about your product are made by "the majority", you get Design By Committee and we all know how well that usually turns out. In good teams, decisions are made by people who are the best fit for making them and there are always conflict-resolution mechanisms in place.

Third, always maintain a measure of stability. Agile is all about adapting to changes quickly, but "change" here has a specific meaning; it refers to evolving requirements, not just any kind of change. You're already pushing a big change by switching to Scrum and you also know that your project requirements are bound to evolve over time, so that means your team will be facing changes on two fronts. Try to keep the rest of their environment as stable as possible. If you really have to introduce changes, do so gradually. Changes in team structure -- such as dividing the team into smaller teams to avoid crowding the daily scrum or merging two smaller teams into a larger one -- can be very disruptive. The lack of stability might easily demoralize your team, so that you end up losing more than what you hoped to gain by restructuring.

So far, I've talked about measures that could be taken by anyone with enough authority to see them through. The last piece of advice I'd like to offer applies to consultants exclusively. It's something that should be completely unnecessary to say, yet it seems that some consultants might need to be told this: be there. Yes, I know that you're just one consultant and you're bringing change to the whole company and you have to go scurrying hither and yon to assess all the risks and ensure success and whatnot. But if you're not there to support your team all they need, they -- and their project -- are going to suffer. They're under pressure from all sides: they have an important project which must succeed and they're expected to make this happen in a new way, using a new process. Be there for your team, listen carefully to their needs and do whatever you can to help them. You're the one who understands how things should be done, so make sure that the team can rely on you. It's not enough to tell them you're there to help them; actions speak louder than words and if you're not there, you'll lose their trust quickly. And it all goes downhill from there.

Sounds hard? Welcome to the real world, where there ain't no such thing as a free lunch.

Reflection Recipes

noreply@blogger.com (Anonymous) — Mon, 23 Feb 2009 21:30:00 +0000

Have you ever seen that cartoon in which Speedy Gonzales guides Daffy Duck across a minefield?

BOOM! There's one! BOOM! There's another! What do you mean you don't know where they are? You haven't missed one yet!

I've been plumbing the depths of .NET reflection lately and it felt just like Daffy's trip: I knew where I was and where I wanted to be, but I hit every landmine on my way there. It's not Microsoft's fault, really. The .NET reflection is a powerful beast and covering every little "gotcha" is a heroic work, one that the guys in charge of MSDN almost pulled off. Nearly everything you'll need to survive is somewhere in the library. If you're trying to do something simple, you'll be fine; if you're trying to pull off something advanced, you'll need a map of the minefield. Here are some of the mines I hit so far.

Attributes: Not What It Looks Like, Honey

It's easy to think of attributes as something that gets attached to other elements of your code when it's compiled. Unfortunately, it's also incorrect. Consider the following code:

[AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = true)]
public class UniqueIDAttribute : Attribute
{
    private static int nextID = 1;

    public int ID { get; private set; }

    public UniqueIDAttribute()
    {
        ID = nextID++;
    }
}

public class Base
{
    [UniqueID]
    public virtual void MyMethod() { }
}

public class Derived : Base
{
    public override void MyMethod()
    {
        base.MyMethod();
    }
}

class Program
{
    static void Main(string[] args)
    {
        UniqueIDAttribute attr;
        attr = (UniqueIDAttribute) Attribute.GetCustomAttribute(
        typeof(Derived).GetMethod("MyMethod"),
        typeof(UniqueIDAttribute));
        Console.Write(attr.ID);
        Console.Write(" ");
        attr = (UniqueIDAttribute) Attribute.GetCustomAttribute(
        typeof(Base).GetMethod("MyMethod"),
        typeof(UniqueIDAttribute));
        Console.WriteLine(attr.ID);
    }
}

I was honestly surprised that the output of this was "1 2" instead of "1 1" I expected. I thought that placing a custom attribute on a method will attach an instance of that attribute to that method. This is not feasible for a variety of good reasons; for example, how do you handle a case of an attribute placed on a member of a generic class? Most of these problems could be worked around with lots of effort, but Microsoft decided to do things the easy way and invest that effort somewhere else.

Here's what happens when you place an attribute on a code element: the compiler takes all the information necessary to construct that attribute and emits it together with that element. When you query the attribute at run time, the reflection code pulls out this information and uses it to construct the instance of the attribute at that moment. As an example, consider the following code:

[AttributeUsage(AttributeTargets.Method)]
public class SampleAttribute : Attribute
{
    public int Value { get; protected set; }
    public string Comment { get; set; }

    public SampleAttribute(int value)
    {
        Value = value;
    }
}

public class SomeClass
{
    [Sample(17, Comment = "Some text")]
    public void SomeMethod() { }
}

Now take a look at the IL for SomeMethod:

.method public hidebysig instance void  SomeMethod() cil managed
{
.custom instance void Sandbox.CSharp.SampleAttribute::.ctor(int32) = ( 
               01 00 11 00 00 00 01 00 54 0E 07 43 6F 6D 6D 65   // ........T..Comme
               6E 74 09 53 6F 6D 65 20 74 65 78 74 )             // nt.Some text
// Code size       2 (0x2)
.maxstack  8
IL_0000:  nop
IL_0001:  ret
} // end of method Base::SomeMethod

As you can see, the method metadata contains custom attribute information that tells the runtime which constructor to call, what parameter to supply to it and which property to set to which value.

Having understood all that, it's no problem to understand why the CustomAttributeBuilder class exists or how to use it.

Non-Virtual Interface: The Dark Secret

Have you ever wondered how it is that you can implement an interface method in a class, without marking it as virtual? It had been bothering me since I learned C#, but I never really had a compelling reason to solve that particular mystery. It was enough to know that it Just Worked. At least, it was until I tried to dynamically create a class and make it implement an interface too.

I'll stop here to make a quick aside for readers who know only C# (and/or Java) and might wonder why I think that interface methods should necessarily be virtual. A virtual method is late-bound: you don't know at compile time which implementation will be invoked when someone calls the method. Because of that, there's a layer of indirection between the method declaration and the method body; the class that defines the body of a virtual method -- either for the first time or by overriding it -- "configures" this layer of indirection to "point" to that particular body. An interface is nothing but a bunch of abstract (and therefore virtual) methods. That's why it seems counter-intuitive to implement an interface method without specifying the "virtual" keyword.

If you try to run the following code, you'll get an interesting exception:

public interface IMyInterface
{
    void MyMethod();
}

class Program
{
    static void Main(string[] args)
    {
        AssemblyBuilder assembly = AppDomain.CurrentDomain.DefineDynamicAssembly(
            new AssemblyName() { Name = "MyAssembly" },
            AssemblyBuilderAccess.Run);
        ModuleBuilder module = assembly.DefineDynamicModule("MyModule");
        TypeBuilder myClass = module.DefineType(
            "MyClass",
            TypeAttributes.Class | TypeAttributes.Public,
            typeof(Object),
            new Type[] { typeof(IMyInterface) });
        MethodBuilder myMethod = myClass.DefineMethod(
            "MyMethod",
            MethodAttributes.Public | MethodAttributes.HideBySig,
            typeof(void),
            Type.EmptyTypes);
        ILGenerator il = myMethod.GetILGenerator();
        il.Emit(OpCodes.Ret);
        Type myClassType = myClass.CreateType();
    }
}

The exception you'll get is TypeLoadException and it'll be complaining about MyClass not implementing MyMethod. If you add the MethodAttributes.Virtual flag to the method attributes, the problem will disappear.

But what happens if you don't want MyMethod to be virtual? That's not really an option. For the reasons I explained above, the method must be virtual. What you might want is to make sure it cannot be overridden. To do that, you have to also add the MethodAttributes.Final flag to the method attributes. That will give you the equivalent of writing:

public class MyClass : IMyInterface
{
    public void MyMethod() { }
}

The funny thing is that you can't declare a method as virtual sealed, yet that's exactly what the compiler does under the hood. I guess it's just one of those trade-offs: either you use a slightly leaky abstraction or you risk antagonizing people by making them learn the underlying concepts thoroughly.

Naming Your Type: Basic Hygiene

Isn't it funny how something as useful as the string type can also cause so much trouble? Sooner or later everyone has to learn to sanitize their strings or risk having their software let the little Bobby Tables wreak havoc on their data:

I must admit, though, that I didn't expect to have to sanitize the name when creating a type at run time. I found out, quite unexpectedly, that TypeBuilder seems to be allergic to types whose names contain a comma. Not that it blows up right away. You can use a name riddled with commas and still be fine when you create the type. What you can't do is use that type as a parent type. If you do, TypeBuilder will blow up with a rather unhelpful COMException saying "Record not found on lookup."

If you google for it, you'll eventually find out that there's a bug in reflection, supposedly triggered by strings that contain any of the characters []*&+,\ in the type name. My experiments indicate that the only problematic character seems to be the comma, but I'm getting rid of all of those characters anyway.

It's a nasty little bug and it's not documented in MSDN. I figured I should warn people about it, since it's such a comma mistake after all.

Metadata Tokens: Remembering Past Lives

There are two stages in the life of a dynamic class: before and after the call to CreateType. In the first stage, you're using the TypeBuilder to define type features, such as fields, properties, methods, nested types, etc. When you're done defining your type, you call the CreateType method and the magic reflection fairy turns your TypeBuilder into a real live Type. But your little Pinocchio can either be a marionette or a real boy, and never the twain shall meet.

In practice, this means that a FieldBuilder you get from calling DefineField on your TypeBuilder can only be used to define the field. Even though the FieldBuilder inherits from FieldInfo, you cannot use it to get or set the field value on the instances of the type obtained by calling CreateType on the TypeBuilder. If you try to do so, you'll get a NotSupportedException, informing you that the "invoked member is not supported in a dynamic module". Same thing applies to other builder classes.

Looking around the documentation, you'll see that the usual workaround seems to be to look up the member by its name in the newly created type. That can get ugly, though. For example, consider what you would have to do to look up a one of the several methods with the same name. The infuriating thing is that you already have the MethodBuilder for it, which is the equivalent of a MethodInfo in its TypeBuilder, so there should be a relatively painless way of getting its counterpart in a live Type.

It turns out that there is a rather painless way of doing it and it's not even a dirty hack. Every type and every member in has a metadata token that identifies it uniquely within its module. If you have the metadata token for a method, you can resolve it into a MethodBase by calling ResolveMethod on the method's Module. The usual way of obtaining a metadata token is via the MetadataToken property defined in the MemberInfo class. That won't work on a builder class, though. You'll get an InvalidOperationException complaining that the "operation is not valid due to the current state of the object".

In order to get the metadata token from a builder class, you have to use a corresponding method in the ModuleBuilder class. For example, if you have a MethodBuilder and want to get its metadata token, you have to call GetMethodToken on its ModuleBuilder. When you put it all together, converting a MethodBuilder for a TypeBuilder into its counterpart MethodInfo for the resulting live Type can be done like this:

public static MethodInfo GetLiveMethod(MethodBuilder method)
{
    return (MethodInfo) method.Module.ResolveMethod(
        ((ModuleBuilder) method.Module).GetMethodToken(method).Token);
}

Clean, concise and probably more efficient.

Generic Parameters: Working Around The Taint

When I was checking out Ruby, I ran into this neat concept of taint. It stuck with me, probably because I read about it in The Pragmatic Programmer's Guide, whose style is rather memorable. And, of course, because the concept is so neat.

What, then, is taint? Besides being "the opposite of tis", 'tis a way of keeping little Bobby Tables out of mischief. The idea is simple: anything coming from the outside user is considered tainted and anything derived from a tainted value is also considered tainted; if taint checking is active, the tainted values cannot be used for anything dangerous, such as SQL queries or host operating system commands.

What does this have to do with .NET reflection? The generic parameters for dynamic types and dynamic methods behave in a similar fashion. Specifically, if you call an otherwise innocent method MakeGenericType on an existing generic Type, you'll wind up with a Type with limited capabilities. Calls like GetField will fail on such a Type.

The workaround is actually documented in MSDN, which is why I saved this particular landmine for the end. If you've already read this far, chances are you're going to read and remember this solution before you actually run into the problem.

Here's what you have to do instead of calling GetField on the "tainted" Type. First get the FieldInfo from the generic type itself, as in:

FieldInfo myField = typeof(MyGenericType<>).GetField("MyField");

Then, get the "tainted" type:

Type constructedType = typeof(MyGenericType<>).MakeGenericType(myGenericParamBuilder);

Finally, do the workaround magic:

myField = TypeBuilder.GetField(constructedType, myField);

Rinse and repeat until you run out of "taint".

Further KABOOM Avoidance

The .NET reflection is really powerful and flexible, but it can be a real pain in the backside if you're not careful. Now that C# 3 is here, you can opt to use the Expression<TDelegate> instead of old DynamicMethod, but the old-style TypeBuilder is not likely to go anywhere soon. I hope that this map will help you get across the minefield without getting blown up often.

Interfacing Outside the Box

noreply@blogger.com (Anonymous) — Wed, 14 Jan 2009 14:29:00 +0000

Happy New Year, everyone! Another hiatus is over and I hope to make it the last one. I've never been the one for New Year's resolutions before, but I guess there's a first time for everything: this year I resolve to post at least once a month.

I gave my self a head start by accumulating a number of topics I want to write about. After some careful consideration, I decided to start off with a bang: by introducing a new design pattern!

Hacking the Language

A while ago, I wrote about design patterns and how they're often a result of having to work your way around language limitations. No language is perfect, so you're bound to run into an annoying limitation sooner or later, even if you're not spending your days coding in Blub. If you're lucky, there's already a design pattern that allows you to hack your way around it. If not, you'll have to invent a solution yourself.

One of such limitations in C# is the boxing that occurs when you cast a value type to an interface. I've wrote about a related topic before, back when I was just learning about value types and boxing in C#. You could probably tell, because I made a spectacular mistake in my speculations.

Safe Deposit Box

In the course of devising a solution for the problem described in Enum Conundrum, I erroneously suggested that a possible solution might have been to have my enum implement the IEquatable interface, had the language allowed it. That, of course, wouldn't have helped. The enum would still have been boxed, although for a different reason.

Why do value types get boxed when cast to an interface? The reason is simple enough: for safety. For example, if you have a local value type variable, it resides on the stack and gets destroyed after the call ends. Without boxing, if you stored an interface reference to that value type instance in an object field, that field would wind up with an invalid reference as soon as the original call is done and the variable goes out of scope.

Of course, you might not care about boxing. After all, boxing was invented precisely so that people could use value types and reference types on equal footing: the alternative to it is all that nasty wrapping code that was so widespread in Java before it incorporated boxing. So boxing is neat, isn't it?

Get Back in Line

When you define a value type, you (should) do it for a good reason. Most of the time, it's because you need a type with value semantics; in other words, you want a type that behaves in such a way that it's not possible to affect one variable of that type by performing operations on another variable of the same type.

The way value semantics is enforced is through memory allocation. Value types are allocated in-line. This means that the memory allocated for a value type instance is a part of the memory allocated for whatever contains that instance. If it's a local variable, the instance is allocated directly on the stack. If it's a field, the instance is contained in the memory allocated for the field's object. Only when it gets boxed does a value type wind up on the heap independently, all by itself.

At times, this is precisely the reason why you'll choose a value type: not so much for its semantics, as for its memory allocation. For example, you might be writing a game in XNA and you don't want the garbage collector kicking in and ruining your frame rate.

Whatever your reasons for wanting to keep them in line, the fact remains that you cannot cast your value types to interface without them getting boxed. What, then, is the alternative?

Generic Static

Fun fact: a static field in a generic class is allocated separately for each closed constructed type of that generic class. What does this mean? Take a look at the following code:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace BeardsEye
{
    class GenericStatic<T>
    {
        public static int SomeVal;
    }

    class Program
    {
        static void Main(string[] args)
        {
               GenericStatic<int>.SomeVal = 1;
               GenericStatic<string>.SomeVal = 2;

               Console.WriteLine(GenericStatic<int>.SomeVal);
               Console.WriteLine(GenericStatic<string>.SomeVal);
        }
    }
}

If you run it, it will print out 1 and 2. Of course, such behavior stands to reason; if you're not sure why, just change the type of SomeVal to T.

Why am I suddenly talking about this, anyway? Because, in some cases, you can use this trick as a substitute for what would otherwise be a dictionary. Specifically, for some uses, you could replace a Dictionary<Type, Something> with:

public static class MyTypeDictionary<TType>
{
    public static Something Value;
}

Of course, this is a far cry from a fully functional dictionary. You don't have the Count property, there's no way to remove values, iterate over them or find out whether a type has an associated value or not. And you can't have an arbitrary number of these "dictionaries".

On the other hand, when you need to associate something with a type and that "something" changes from type to type, this technique is just perfect.

Fun With Functions

Now we finally get to the solution itself. Suppose you have an interface IConfusticatable, for all types that can be confusticated to a certain degree:

public interface IConfusticatable
{
    double Confusticate(int degree);
}

You want to be able to confusticate classes that implement this interface, but you would also like to confusticate value types, without boxing them. You could use the following trick:

public static class Confusticator<T>
{
    public static Func<T, int, double> Confusticate = ((what, degree) => ((IConfusticatable) what).Confusticate(degree));
}

What do you have to do in your value type? Not much:

public struct MyStruct
{
    static MyStruct()
    {
        Confusticator<MyStruct>.Confusticate = ((me, degree) => me.Confusticate(degree));
    }

    public MyStruct(double val)
    {
        Val = val;
    }

    public double Val;

    public double Confusticate(int degree)
    {
        return Val + degree;
    }
}

How do you use this when you want to confusticate something? Like this:

MyStruct myVar = new MyStruct(17);
Console.WriteLine(Confusticator<MyStruct>.Confusticate(myVar, 3));

I'm sure that, at this point, you're thinking "So what good is this? If I already know that myVar is a MyStruct, I can just call its own Confusticate method directly."

This, of course, is perfectly true, unless you're writing generic code. In that case you don't know beforehand what your type parameter will be.

Warning: May Contain .NUTS

A few words of caution here. The proposed implementation of MyStruct relies on a static constructor, which is supposed to "register" the confustication lambda with the Confusticator. The problem with this is that it's a bit tricky to make sure that a value type static constructor is invoked.

If you dig around the C# Annotated Standard, you'll find some fascinating incompatibilities between different standards and their respective implementations.

According to the C# Standard, a static constructor for a value type is executed only when the first of the following occurs:

An instance member of the struct is referenced.
A static member of the struct is referenced.
An explicitly declared constructor of the struct is called.

On the other hand, the CLI Standard requires execution only when the first of the following occurs:

A static member of the struct is referenced.
An explicitly declared constructor of the struct is called.

To further complicate the matters, the CLI Standard does not allow execution when an instance member of the struct is referenced. You'll note that this is in direct conflict with the C# Standard.

If you actually test what happens when you run a C# program on CLR, you'll see that the static constructor is called as soon as the first of the following occurs:

An instance method of the struct is referenced.
An instance property of the struct is referenced.
A static member of the struct is referenced.
An explicitly declared constructor of the struct is called.

However, the static constructor is not called if you reference an instance field of the struct. Not only does the CLR implementation fail to conform to either standard, it does so in a way that seems spectacularly arbitrary. I'm sure there are some perfectly valid -- if arcane and obscure -- reasons for this.

All in all, it would be a lot more reliable to stick the "registration" code in some initialization method that you know will be called. That's what you would have to do anyway, if you wanted to register a confustication function for an enum, for example.

Finishing Touches

What if your IConfusticatable interface has more than one method? There are several possible solutions, but the simplest one would be to give your Confusticator<T> one static delegate field per IConfusticatable method.

Speaking of Confusticator<T>, if you find it too ugly to write Confusticator<MyStruct>.Confusticate(myVar, 3), you can use type inference to introduce some syntactic sugar:

public static class Confusticator<T>
{
    public static Func<T, int, double> Confusticate = ((what, degree) => ((IConfusticatable) what).Confusticate(degree));
}

public static class Confusticator
{
    public static double Confusticate<T>(T what, int degree)
    {
        return Confusticator<T>.Confusticate(what, degree);
    }
}

With that, you would be able to simply write Confusticator.Confusticate(myVar, 3) and have the compiler infer the type automagically.

Share and Enjoy

Well, that's all for now. For those of you who have a legitimate need to avoid boxing your value types, I hope this proves useful. For the rest, I hope that you found this an interesting trick.

I'm planning to write more recipes for getting stuff that C# doesn't have, such as multiple dispatch, so stay tuned!

"Encryption Chip" Will Not End Piracy

noreply@blogger.com (Anonymous) — Thu, 24 Jul 2008 20:40:00 +0000

Nolan Bushnell is full of it. There, I finally got that off my chest. It's arguably acerbic and rather rude, but it needed to be said. You have no idea how hard I've tried to avoid saying it. After all, he's the founder of Atari and a historical figure unto his own. He deserves certain respect for that.

The first time Nolan Bushnell claimed that the "encryption chip" will end piracy, I exercised due restraint. His statement reverberated all over the Internet, causing reactions that ranged from mild skepticism on one end of the spectrum to derision and disgust on the other.

So why am I writing now, more than two months later, if nobody believed him in the first place? In other words, why am I beating a dead horse? Partly, it's because he did it again and it pisses me off. Mostly, though, it's because I'm rather interested in copy protections and security; it's sort of a hobby of mine.

The most important lessons you learn in those two fields is that no protection is perfect and every solution spawns a new class of problems. This means that there will never be one final (technical) solution to the issue of piracy; there is no silver bullet. The experts from both fields are locked in an arms race with their adversaries. Once you've learned that, you'll have no problem recognizing that Nolan Bushnell is really just flogging his merch.

However, the issue runs deeper than that.

Copy Protection and Security

When I referred to copy protections and security I said "two fields", even though one can be considered a subset of the other; after all, copy protections are supposed to prevent the unauthorized use of software. Even though this is technically true, there are some drastic differences between the two.

An important difference is the level of cooperation from the users. When it comes to information security, the users actively cooperate with the protection systems, because it's in their best interest. You don't give access to your bank account to all your friends, do you?

On the other hand, copy protections often clash with the users' interests. Some of these interests are illegal, such as downloading a commercial game for free. But other interests are quite legal and legitimate. You added more memory to your computer? Odds are you might have to reactivate your Windows.

Another important difference is that a copy protection has to protect an application that lives on the user's computer. Unless we're talking about an MMOG, there's no server counterpart that executes a critical piece of code, without which the game can't work.

When you put those two things together, it becomes obvious why you can't make a perfect copy protection: you're relying on cooperation from a user that has complete control over his copy of your content or software. If that user doesn't want to cooperate, the best you can do is delay him. Even unbreakable ciphers won't help you, because sooner or later you'll have to decrypt the content and, when you do, the user will nab it.

But what if you could alter these conditions? You could make sure that there's a critical part of an application that executes somewhere where the user doesn't have control over it: that's what MMOGs do. The other option is to take away the control from the user.

Trust Controversy

Enter the "trusted computing". The first time I heard of it was back when Microsoft was touting Palladium. Back then, it sounded like a bad pun: a company found guilty in an antitrust lawsuit is proposing to build a "trusted computing platform" for its users. The irony was not lost on anyone and it provoked some enlightening responses from security experts.

Then, since nothing really seemed to happen and we didn't all suddenly wake up in some digital equivalent of 1984, I lost track of this topic for a while. I forgot about it until Nolan Bushnell started his TPM hype. A quick search engine query revealed that TPM stands for "Trusted Platform Module" and that it's the central component of "trusted computing".

What, then, is the so-called "trusted computing"? It's a technology that encompasses the following concepts:

Endorsement key is a cryptographic key pair unique to one computer. The chief use for it is to prove the computer's identity.
Secure I/O makes sure that the communication between the user and their software is secure and cannot be intercepted or altered.
Memory curtaining protects those parts of memory that contain sensitive data (such as cryptographic keys) from unauthorized access, even by the operating system itself.
Sealed storage binds data to the specific platform -- both hardware and software -- so that you cannot access it from any other platform.
Remote attestation allows authorized parties to detect changes to the platform configuration in order to make sure that they meet the expected parameters; in other words, to prove that nobody tampered with the platform.

That's just a brief summary, to give you an idea of what we're talking about here. If you want more information, I recommend that you start at Wikipedia and then go on directly to the Trusted Computing Group site.

So, the core idea is to make computers more secure, by ensuring that no "untrusted" code has access to your stuff. At least, that's supposed to be the core idea. Unfortunately, there has been a great deal of confusion about the word "trust" in "trusted computing". Specifically, who is supposed to trust whom?

If you read Bruce Schneier's essay on "trusted computing", you'll notice that there's a good deal of controversy and confusion surrounding the issue. As one commenter so aptly put it, the only one not trusted seems to be the owner of the computer.

All Your Base

Each of the five concepts of "trusted computing" addresses a real security problem:

Endorsement keys would be used to mitigate spoofing concerns in secure transactions by establishing the identity of each party involved.
Secure I/O is supposed to avoid security breaches through techniques such as keylogging.
Memory curtaining would make sure that sensitive information, such as cryptographic keys, is not allowed to "leak" somewhere where it could be extracted by malicious parties.
Sealed storage would do a similar thing for sensitive information in non-volatile storage.
Remote attestation could help network administrators easily detect intrusions and attacks on their machines.

Yet, after a closer look at them, it becomes evident that there's plenty of room for abuse. Imagine, for example, a system that enforces specific usage policies on your data:

It would use sealed storage to bind that data to a particular application or set of applications that you're allowed to use on that data.
It would employ memory curtaining to make sure you cannot extract that data directly from memory.
It would use secure I/O to make sure you cannot intercept it on its way somewhere else.
It would use remote attestation to report if you tamper with any part of the system.
And it would clearly identify you as a "culprit" to whoever is interested in enforcing those policies, if it possessed both your personal information and your endorsement key.

Is there any kind of usage policy that springs immediately to mind? There are two, actually: DRM and vendor lock-in. Ross Anderson describes several ways to abuse TC in his FAQ. Richard Stallman dedicates a whole chapter to this topic, in his book "Free Software, Free Society"; although slightly reminiscent of Book of Revelations in tone, it offers some very interesting insights.

Another interesting aspect of "trusted computing" is that it actually raises the stakes when it comes to information security: imagine a worm that successfully exploits a bug in the supposedly secure OS code to install a "trusted" rootkit? Talk about irony.

Pirates vs. Ninjas

Getting back to the original topic, does this mean that Nolan Bushnell is right? Is his "stealth encryption chip" really going to send all the pirates to the Davy Jones's Locker? Not by a long shot! Remember, if the software in question does not have some critical code running on some computer under control of some "authority", you can eventually break its copy protection.

When it comes to policy enforcement, the most important part of the "trusted computing" is the remote attestation. This is the way to ensure you won't tamper with the policy enforcement code. Incidentally, it requires you to be online. Now back up a couple of months and remember what happened when BioWare tried to pull that trick on its players.

If you believe that pirates can't hide behind this forever, think again. There are numerous valid reasons to resist the attempts to introduce an artificial dependency on Internet connection into software and all those reasons boil down to one: the connection is not always available, yet the artificial nature of the dependency means that the software doesn't actually need it to work properly.

Besides, people are not yet convinced that "trusted computing" will actually make things better. Plus, there are all sorts of concerns about privacy and also about practicality of the whole approach. Still, the Trusted Computing Group has been formed, the commercial motivation is there and "trusted computing" will keep rolling, until all doubts and concerns have been dealt with, one way or another.

What, then, is the worst blow "trusted computing" could deal to pirates? Indulge me a bit, as I let my imagination run wild and explore a "what-if" future.

Crack Dealers

Back when I was a little kid, learning what makes the cute, little Spectrum 48 tick, pirates were selling games on audio tapes. Today, pirated games are free. They are cracked for free, by enthusiasts; they are uploaded for free, on sites that survive on advertising or donations; and they are downloaded for free. I can still see some pirates in the streets, selling CDs and DVDs, but I'm sure they won't be buying any Ferraris with that money.

Fast forward to the time when "trusted computing" is in its full swing. To crack protections, pirates need very specialized software, maybe even some hardware, and a lot more effort than before. More than ever, piracy is something that only the select few can do.

However, it is also more lucrative than ever. As the usage policies are enforced more rigorously, the multitudes who used to obtain their entertainment for free now have to go and buy it. The big companies take advantage of that and the prices are even higher than before. You can buy an overpriced game directly from its publisher; or you can take a chance and go buy yourself a pirated copy from a local "software crack dealer". It's illegal, sure, but it's a lot cheaper and you can afford to buy a lot more.

Suddenly, pirates are not your everyday enthusiasts anymore; instead, they're rich criminals. They have bodyguards with guns. They have shady lawyers. They have money laundering enterprises and fake fronts and lots of connections. They know powerful people. In your efforts to eradicate a problem, you managed to make it mutate into something worse.

If this seems improbable and exaggerated, that's okay: I don't believe it's likely to happen. My point is that you should always be on the lookout for unintended consequences. It would be nice if, just for once, we asked ourselves where we're going before we get there.

Update: Firefox 3 Works After All

noreply@blogger.com (Anonymous) — Sun, 20 Jul 2008 23:13:00 +0000

Consider this a deep breath before I plunge into some more serious blogging. It turns out that Firefox 3 works rather well, after all. Not that I doubted it, but I couldn't really try it out until all my favorite add-ons were updated and relatively stable.

The biggest difference I've noticed so far is that it doesn't do its famous hog-the-CPU trick on pages such as Google Spreadsheets. Now that is a big relief for me; I absolutely hated that particular bug.

Make no mistake: I didn't change my mind about my first experience with Firefox 3. I still believe the developers should have allowed their users to do one of the following:

check the add-on compatibility during the installation
revert to Firefox 2
run Firefox 2 and Firefox 3 side by side

But, hey, at least it works now. And I've only confirmed that I'm a late adopter by nature. As if I really needed to confirm that: I've lived in Chile for nine years before I decided to try Carménère.

Firefox 3: A Five Act Tragedy

noreply@blogger.com (Anonymous) — Wed, 18 Jun 2008 01:46:00 +0000

Act I: The Hype

I first heard of the Firefox Download Day at the office, from a co-worker. It wasn't a dark and stormy night, however much we all wished it to be -- the A/C was on the blink yet again.

Naturally, my curiosity was piqued. I've been using Firefox for years now and I've grown used to it and a bunch of add-ons for it. Sure, it had a couple of annoying bugs, but I could live with those. Of course, a new and improved version was likely to get rid of them, which would have been incentive enough to check it out.

So I went to the "Spread Firefox!" site to find out what the fuss was all about. When I found that Mozilla not only planned to give us all a new version of my browser of choice, but also establish a Guiness World Record, I was hooked. I pledged right away and also scheduled a reminder in my Google Calendar, to make sure I wouldn't forget.

Act II: The Outage

It seems childish in retrospect, but I got really excited. Not only was I going to get my hands on the hottest piece of software on the Web, but I was also going to be a part of the effort to set a Guiness World Record. And I wasn't the only one around: a few other co-workers were as excited about this as I was.

When the big day finally came, I made sure I was busy working to keep my mind off the wait. The time passed quickly enough and when the clock was finally done ticking its way to 1 pm, I went back to the Firefox site. That's when I got my first surprise: the site was down.

It's not unusual for a site to be brought down to its knees by overwhelming interest, but I would have expected Mozilla to be prepared. After all, they were the ones who did their best to attract all that attention. Why make a fuss if you can't cope with the results?

After a couple of hours of doing a Wile E. Coyote impression, Mozilla finally got its stuff together and I was finally able to download my copy. Once it was on my disk, I wasted no more time and clicked through the install impatiently. At last, the moment of truth was at hand: I launched Firefox 3!

Act III: The Incompatibility

The first thing my new Firefox did was to check my add-ons. It then proceeded to notify me that less than half of them had updates compatible with version 3 of Firefox. That left more than half of my add-ons disabled.

Being a coder means that an essential part of your job is to imagine worst cases for your code, so
I generally try to be an optimist about everything else, just to balance things out. Therefore I surmised that the Mozilla site was still experiencing difficulties. Did I mention that my optimism is rarely founded?

I spent some time fumbling with the new user interface, only to find out that the developers didn't stop at adding the add-on search into Firefox itself. They also decided to remove the link that took the users directly to the add-on page. I ended up checking manually and the add-on page was working quite well, thank you.

That's where I felt Mozilla managed to set a record, although it wasn't the one they were hoping for: I went from enthusiastic to disappointed in less than 5 minutes since I installed their software. What were they thinking?

Pretend you're in charge of developing a rather successful software product. One of the key advantages it has over the competition is its extensibility. There's a huge variety of add-ons for it and they do lots of different things, ranging from silly to extremely useful. Even though your users might be dissatisfied with some aspects of your software, they would still be reluctant to change to a product that doesn't have the add-ons they use. So what do you do when you decide to develop a shiny new version of your product?

The fashionable answer seems to be "break the add-ons that worked nicely with the older versions", even though the history teaches us the opposite.

Okay, I understand that you can't always maintain compatibility. Sometimes, preserving compatibility is in direct conflict with a higher-priority design goal and you just have to go ahead and break stuff. What you do in those situations is to try to minimize the damage, which is what Mozilla tried to do. But what do you do about the stuff the remains broken?

Here's what you do: you make your installer check the compatibility and report to user before installing the new version and breaking your user's stuff.

Act IV: The Retreat

Once again life conspired to remind me why I'm a late adopter. After a minute or two of staring at the list of things I wouldn't be able to use with Firefox 3, I decided to go back to version 2. A routine check of my download folder revealed that I apparently got rid of the installer at some point in time. No big deal, I can download it from the Firefox site, right?

Wrong! There is no link to Firefox 2 anywhere on the whole page. No matter how hard you try, you won't be offered a chance to turn back. And I tried very hard, trust me. Among other things, I tried typing "firefox 2" into the search box. That particular attempt rewarded me with zero results. I didn't even get a link to a press release or release notes for Firefox 2. Zilch. Firefox 2 has been exterminated.

Despite all its pictures of birds and balloons and a smiling Sun, the message Mozilla is sending its users is loud and clear: "Assume the position!"

In the end, I found the download link on www.firefox2.com, a site that features a prominent notice that it isn't associated with Mozilla in any way. It worked for me, but it probably won't work for you. As I'm writing, I'm also looking at the new content of the site, where every occurence of "Firefox 2" has been replaced with "Firefox 3", except the site URL, of course.

Just in case you really need to go back, the solution is to grab the Firefox 3 download link and change it manually to request the version 2.0.0.14. For example, the correct link to download the English (US) version of Firefox 2.0.0.14 for Windows is:

http://download.mozilla.org/?product=firefox-2.0.0.14&os=win&lang=en-US

Act V: The Repose

Having successfully overcome the unexpected difficulties, I decided it was time to wrap things up and go back to work. I installed Firefox 2 over its younger sibling and launched it. My optimism was proven unfounded yet again: Firefox neatly crashed when it tried to load my saved session.

I tried a few things before I stopped fighting the inevitable and uninstalled Firefox. Hoping fervently that the problem would be solved this time, I reinstalled it and launched it.

By this time I was more than a mite miffed or a tad testy: I was positively pissed off. In the spirit of their decision that I won't need version 2 ever again, Mozilla obviously didn't bother to test the downgrade process.

Fortunately, it turned out that no drastic measures were necessary. I was back to Firefox 2 and all my add-ons worked again.

Epilogue

While writing this post, I stumbled upon this article that explains how to make older add-ons work with new Firefox. In short, it describes a quick hack that makes Firefox skip the step where it checks for add-on compatibility. The article was about version 3.0rc1, so I don't know whether this will work in the final release. I didn't try it and I don't intend to. I prefer to wait until all the add-ons I use are updated. Having a new browser that's shiny and sleek is cool, but being able to use it my way is way cooler.

Piracy is Here to Stay

noreply@blogger.com (Anonymous) — Thu, 15 May 2008 19:28:00 +0000

"With great power comes great responsibility." Wise words, Uncle Ben, but someone at BioWare wasn't listening. For those who hate following links, the short version of the story is that BioWare was going to implement draconian anti-piracy measures in Mass Effect and Spore for PC. In the end, they won't be doing it, but it wasn't for the lack of trying. Indeed, I tend to agree with Penny Arcade when it comes to predicting the next step in the war on piracy, a war that has a long history of going overboard.

What's interesting is the widespread reaction among the gamers:

Not only does this not make me want to buy the game, this makes me want to download a pirated and cracked copy even more

Funny, isn't it? I mean, when someone mentions using RFID for shoplifting prevention, you'll hear all sorts of privacy concerns and doubts about usefulness of such a system, but you won't hear people say they'll go shoplifting in protest.

So what is it that makes us perceive and react to piracy in such a different way? Let's take a look at some of the factors involved. Be warned, though: there's no magical happy ending to this story. Whether the hero defeats the villain and gets the girl remains to be seen.

Thou Shalt Not

Back where I originally come from, some 15 years ago, shops didn't have any shoplifting prevention mechanisms besides humans and, if the shop was large enough, mirrors. We all knew shoplifting was wrong, that it was stealing, that it was crime. We also knew that getting caught would have serious repercussions. Yet there were kids who did it. Not out of necessity, either. They did it for fun. It was challenging. It was a game.

Where am I going with this? There are several points I'm trying to make here. First of all, crime has a lot to do with perception. Sure, there's law, with its letter and its spirit and a bunch of lawyers and politicians to play with it. If you break the law, you're a criminal and that should be it. And yet, none of those kids saw themselves as criminals.

Second, it's in people's nature to do as they wish and rules be damned. You can try to educate them, to impress upon them that such behavior is wrong, but there will still be those who will do it and for a variety of reasons, too.

It might sound like I'm trying to justify piracy, theft and crime in general. I'm not. I'm trying to point out that they will always exist and that reasons for it are not as monolithical as we would like to believe. After all, it would be a lot simpler if people committed crimes because they're evil, but the world doesn't work that way. And if we want to do something about a problem, we have to understand the cause.

Can't Touch This

Every time I rent a DVD, I have to watch one of the MPAA's anti-piracy videos. There's no way to skip it, of course, so each time I'm freshly reminded of the first and foremost difference between stealing and piracy: intangibility. In order to steal a car, a handbag, a television or a DVD, just like in the video, you have to reach out and grab something tangible. There might be a lot of objects like that one, but there is only one of that particular object and it's not yours. Taking possession of it is clearly identifiable as a crime.

Things are not as intuitive with music, movies and software. Sure, if someone invites me to their house to watch a DVD with them, it's not a crime. They're sharing their property with me of their own will. If they offer to give me a lift in their car, it's the same sort of thing. But you can't copy a car like you can a song, can you? You can't parody a car, either, or quote it.

Intellectual property is a complicated and muddled concept and requires an equaly complicated and muddled definition of fair use. With the massification of the digital content and media, things got even more complicated and muddled, which brings us to the chaos we're living nowadays.

And chaos it is. How else would you call it when in one corner you have people who demand that all the digital content be free, as in free beer, and in the other corner you have giants like RIAA suing random individuals?

The Age of Middlemen

So far we've looked at some of the factors causing the piracy. But what about the effects of it? Organizations like MPAA and RIAA are claiming that they are losing money. Publishers like EA are complaining about the same thing.

Let's get one thing clear from the start: developers need game sales to survive. They develop games for living and their money comes from sales, just like your money comes from the job you're doing. There is no doubt that if you install and play their game on your computer without buying it, you're using the products of their work without them receiving your money for it.

Notice, however, that I haven't mentioned publishers in previous paragraph. What does RIAA stand for? Recording Industry Association of America. The keyword here is "recording". The people RIAA represent are those who make money by selling sound recordings of music. That used to be very straightforward: they used to sell records.

Vinyl records are a bitch to duplicate. As long as they were the medium for sound reproduction, the recording industry could control the music industry by controlling the medium itself. The advent of magnetic tapes brought the first crisis. The big guys freaked out, tried to defend their territory and lost. That was the major turning point in the history of copyright.

Evolution

Notice how MPAA is making a lot less fuss than RIAA? You know why? Their industry kept evolving. Sure, nowadays you can watch DVDs on your HDTV in your home-theater, but nothing beats seeing the very first screening of the long-awaited movie in your theater of choice. Besides, everyone and their grandmother will have seen the movie before it comes out on a DVD.

The first important factor here is that movie industry still has a meaningful experience to offer to their customers, beyond merely distributing the medium on which the content is stored. The second important factor is that there's ample segmentation. You can choose how much you care about a movie: will you go to a theater or buy a DVD? Or you might rent it before deciding whether to buy it. Or you might just wait for it to come out on cable. Plenty of options out there.

RIAA, on the other hand, got stuck. They just sat there while their golden goose spread its wings and flied away. Well, they didn't just sit. They kicked and screamed and made a hell of a fuss. They're still doing it, in fact. But they failed to adapt and are paying the price for it.

It's something I realized when I went to a Dream Theater concert in March this year. These guys didn't only play their instruments and sing their songs. No, they staged such an impressive multimedia spectacle, that I didn't even mind the fact that they didn't play most of my favorite songs. After a show like that, which in itself must have made them a pretty penny, they could've sold me their newest album without needing the middleman at all.

Or could they?

Eye of the Beholder

Piracy, like I said, has a lot to do with perception. You can call it culture or mindset or education, but the fact is that I don't know many people here in Chile who buy their music. If you look at it from their point of view, the reasoning is simple: why should they? It's out there on the Internet, for free. We all use MP3 players these days, so why should anyone pay for an overpriced lump of plastic that later you have to insert in a drive and rip and transfer to your player before you can listen to it comfortably?

Musicians, that's why. People who make that music need the money. Then again, the record industry screwed things up on that front too. The public perceives them as people who take huge cuts of profits and exploit the authors. It doesn't make piracy right, but it makes it justifiable in people's eyes. We come again to perception.

And perception is a lot bigger problem than most of us would like to admit. Could Dream Theater really have sold us their music directly, without going through the recording industry? Why would we buy it, when we're so accustomed to getting our music for free?

Detox: Rehab or Jail?

A habit is formed. How do we break it? MPAA seems to think the answer lies in the propaganda. The effects remain to be seen, but I personally doubt it will have any significant effect. RIAA, on the other hand, thinks the answer lies in the law enforcement. We've all seen how that's working out so far and I think we can agree that the idea is laughable.

What about software? Specifically, what about games? The software industry in general is not as monolithical as recording or motion picture industry. For example, applications targeted at big corporations don't have to worry about piracy too much, but their target is not as big as the home user market. Each type of software has its own worries, but games are in a particularly hairy situation.

The problem with the games industry is that it resembles the recording industry. As the saying goes, the developers make the game and the publishers make the money. They're not limited to being middlemen, though. Often they also finance the development, which gives them a bit of versatility when it comes to surviving the paradigm shifts. However, the fact remains that the online distribution channels will be making the publishers' role as middlemen increasingly obsolete as time passes. Of course, online distribution brings its own middlemen, as casual game developers are discovering.

Safe Hex: ACME Copy Protection

So how does the software industry fight piracy? We geeks tend to believe that everything can be solved by technology. Thus the copy protection mechanisms.

Now, wiser geeks know that technology is not an answer to everything. As Fravia used to teach, there is no copy protection that cannot be broken. The one time I thought I found an exception to that rule was when I used Kali. I was wrong, of course. The program itself is just a client for the centralized service and, as such, it can be copied as freely as you want. It's semantics, I know, but it's important.

The reason why it's important is because that solution doesn't apply to most games. The most notable exception are the MMOs. Incidentally, it's a pretty important factor for the popularity of this genre among the developers.

The rest of the genres have to choose whether to use copy protection mechanisms or not. If they do, they have to decide how strong to make it. Unfortunately, strong often means "problematic for the legitimate customer" along with "difficult to circumvent". This is what creates reactions of outrage, such as the reactions to the abortive attempt to "secure" Mass Effect and Spore.

No Silver Bullet

Okay, so the neither the propaganda nor the law enforcement nor the technology solve our problem. What's the answer, then?

Unfortunately, nothing worth doing is ever easy. First of all, there is no silver bullet. There's no magical solution to this mess. You can bet that the real solution won't bring you quick bucks and instant happiness.

I'm no expert on these matters and I can't say with certainty how to deal with this. However, I have a hunch and I'm willing to bet it's a good one: nurture the market.

Take the situation here in Chile. For one thing, games are outlandishly expensive. It's not just the price itself, it's how that price compares to people's earnings and other products. Second, there are too few stores selling games and their selection usually leaves a lot to wish for. There are too many obsolete games and too many crap games and too few hot items. Third, the same comments apply when it comes to renting games. Still, there are people who buy games. We do it because we like those games, we appreciate the effort it took to make them and proud to own them. But there's a lot of room for improvement.

What about countries like United States? Adapt. MMOs are just one trend. The Sims games are innovative in that they introduce a social component and allow people to create whole communities around them. Episodic content is another idea that has yet to be fully explored. Who knows what interesting new idea will come along next?

Above all, nurture the market. I don't know any comic book fan that opted to photocopy a comic book instead of buying it. They love comic books. They're quite fanatical about them. Games need the same kind of fans. And they sure as hell won't get them by treating their customers like criminals and making games even harder to install.

And Yet It Moves

So, what, piracy is here to stay and we have to learn to deal with it? Is that it? All this writing and I have no groundbreaking solution?

Yes, that's more or less it. I did warn you at the beginning that the future is still uncertain. Yet I'm sure it's not a dark one. Consider the fact that Sins of a Solar Empire has no copy protection and doesn't seem to need one. Consider the fact that casual games are successful enough to make Rockstar VP nervous.

Are "hardcore" PC games a dying market, then? Not at all. There will always be market for them and, if the industry does its homework, it won't be just a niche market. Though the lessons to learn may be hard, failing to learn them won't be fun, neither for the industry nor for the gamers.

Assassin's Creed

noreply@blogger.com (Anonymous) — Mon, 12 May 2008 22:36:00 +0000

How can you know whether a game is good without playing it? You can't, really. It wouldn't be fun if you could, right? Fortunately, there are people out there who play games and then tell you about their experience. Some of them do it professionally, some of them just because they think the world really needs to know about those games. Assassin's Creed is one of those games that are good enough to make me want to join the latter group. However, unlike most such games, it also has defects that appeal to my inner ranter. Perfect blogging material.

Just to make it clear, this is not a traditional review, rich in detail and sprinkled with screenshots. My goal is not to give you a detailed description of what you experience as you play Assassin's Creed. If that's what you're looking for, then I recommend the Gamespot reviews, for Xbox 360, PS3 and PC. What I'm focusing on here are the strengths and weaknesses of the game. The idea is to help people who never played it before and people interested in game design.

Still with me? Okay, let's delve into Assassin's Creed.

Stunning Visuals

You know how games always look a lot better in reviews and trailers than when you actually play them? I expected the same to happen with Assassin's Creed. To say that I was surprised is a huge understatement. This game is simply beautiful.

My PC has a dated CPU -- the installer warned me that I don't fulfill the requirements -- but my nVidia 7900 GTX more than compensates for it. Even so, I can't configure the graphics to maximum quality and detail. Or rather, I could if I didn't mind playing a slideshow instead of a game. As it is, the graphics are still stunning.

Everything is richly textured and beautifully illuminated. Each city has its own personality. The devastating aftermath of war in Acre makes itself evident not only in its burned houses and damaged city walls, but it also permeates the very atmosphere of the city, thanks to the greyish lighting that stands in contrast to golden colors of Damascus and Jerusalem.

The animations flow smoothly and every now and then you'll get a very dramatic angle that showcases your prowess at swordplay during a fight with guards. Of course, sometimes the game will get it wrong and you'll get a spectacular shot of a tree or a merchant stand between you and the camera.

Another excellent detail is the blurring effect the game applies when you lock on a target. It's supposed to make you feel like an assassin focusing intently on his victim and it does a great job.

Parkour

When I first saw the word "parkour" my reaction went along the lines of "huh?" It turns out that most of us know of it as "free running". It also turns out that it's not the same thing at all. I won't pretend I'm an expert and, for that same reason, I won't explain the difference. You have Wikipedia for that and you can blame them for any eventual error in it. Suffice it to say that the gameplay of Assassin's Creed incorporates parkour in a most enjoyable way.

I remember that one of my friends once proposed to make a game about free running. We shot down that idea for various reasons -- where the hell were we going to get money for an AAA game anyway? -- and one of them was the fact that we thought it wouldn't be interesting on its own, that it needed something more. I still believe that, especially after playing Assassin's Creed. There's something immensely satisfying about ending your crazy race over rooftops with a spectacular pounce and stab that brings your hapless victim down with a startled, strangled cry.

By the way, if you're into game development, I highly recommend reading this Gamasutra article on parkour in Assassin's Creed and Crackdown.

Music and Sound Effects

When I heard that Jesper Kyd composed the soundtrack for the game I had high hopes and I wasn't disappointed. I personally think that he didn't outdo his work in Silent Assassin, but the score is still great: subtle enough not to intrude on your experience, yet dramatic enough to establish the appropriate mood.

I also liked the way NPCs contribute to the game's atmosphere: preachers, beggars, merchants and passers-by all have something to say and they do it in a way that successfully recreates the bustle of a living city. Of course, they have their limitations. If you start paying a lot of attention to them, you'll find them repetitive; but if you focus on your mission, like an assassin is supposed to, they'll provide a rich auditive tapestry to serve as a background.

Good Plot

I firmly believe that games are essentially stories created, in part, by players. Whenever you find yourself gushing with enthusiasm while you describe a match of Unreal Tournament to your friends, you're telling them a story. That said, games don't necessarily need a plot. But if you're going to play an assassin, it's good to have something to motivate you to kill your victims, right?

I'm usually quite critical about plots. It comes from being a fan of books in an age where most people read only newspapers or technical books, if that. I am glad to say that Assassin's Creed has a rather good plot. It's interesting and it's not shallow. Not one among the characters is what he or she seems to be at the first glance. There are layers to be peeled and they reveal that the world around you is not black and white, even though it would be a lot easier on your conscience that way.

Altaïr himself, the character which you control, is a decidedly unpleasant person at the beginning, not likeable at all. You might think it's dangerous gamble to do that to the character with whom you're supposed to identify, but it works out surprisingly well.

Major Cliffhanger

Few are the works that can get away with a cliffhanger. Endymion pulled it off and lived to tell (the rest of) the tale. Assassin's Creed tried to pull it off and excelled at pissing me off. And everyone else, judging by the reactions.

Yes, it's true that almost all of the mysteries that plagued you during the game are resolved at the end. But that's not enough for a satisfying ending. I don't know about people at Ubisoft, but I like closure at the end of the game. It doesn't hurt the sequel at all, just look at the Sands of Time. Here's an idea, Ubisoft: go read Jim Butcher's article about story climaxes. Just in case it doesn't help immediately, the ingredient you're missing is what he calls "resolution".

Cheesy Dialogue

Like I said, I'm a fan of books. That makes me spoiled when it comes to things like plot, dialogue and character development. I'm aware of that and I try to keep it under control when I'm criticizing a game. After all, games are a different medium and one of the worst kinds of game designer is the failed writer.

That said, I have to draw the line somewhere. In case of Assassin's Creed, I drew it here:

Rafiq: He must be stopped!
Altaïr: That's why I'm here.

I don't mind a certain amount of cliché in games. I don't require voice actors to be perfect. But some of Altaïr's lines are way too cheesy to tolerate. It doesn't help that Philip Shahbaz, the voice of Altaïr, seems to have exactly one tone of voice. I guess it's supposed to sound arrogant and menacing, which suits Altaïr's personality admirably, but even so, you cannot apply it to everything you say. Sooner or later, your players start perceiving you as sulky instead of menacing.

To be fair, I should state that not all of the dialogue is cheesy. Al Mualim has some excellent lines and his voice, Peter Renaday, did a very good job.

In-yer-face Interface

Just as I started with the most gushing praise of the best features, so I kept the worst for the last. Many reviews complain about the repetitive nature of the gameplay, but I don't find that to be such a big problem. Solving it would have been quite costly and the developers did their best to mitigate it by offering the player a variety of investigation missions and giving him the liberty to choose which to play and which to skip.

The worst problem, however, comes from some monumentally bad decisions that were perfectly avoidable. All of them have to do with the interface, but the crown jewel among them is the decision to give the player the freedom to move during the cut scenes.

Picture the following situation: After doing lots of investigative work that helped you decided where, when and how you'll assassinate your victim, you finally have him in sight. He's talking to someone else and nobody is paying any attention to you at all. You're free to move, so you start creeping towards his back. You're in position, the moment is perfect and you press the button. And nothing happens. You press the button repeatedly, muttering "stab him, dammit!" It's useless. You're in a cut scene and nothing you do will have any effect. The developers gave you a completely useless freedom and the only thing they achieved is to confuse you.

And they took away a number of very useful freedoms from you, such as the freedom to skip a cut scene or a tutorial. It's an old lesson, but people at Ubisoft haven't learned it. They didn't let you skip the cutscenes in Prince of Persia and they don't let you do it in Assassin's Creed either. Come on, people, it's a Twinkie Denial Condition!

They also took away your freedom to load and save as you wish, which is another Twinkie Denial Condition. While I appreciate that the game saves automatically whenever I finish a task or get to a checkpoint, I still hate it that I have to let the guards kill me if my assassination attempt didn't go the way I wanted, so that the game will reload the last save.

Then there's also the matter of quitting the game or switching the profile. To exit the game, I first have to exit to the Animus, then exit from the Animus, then "quit" the game, then select a profile (which one? doesn't matter, I want to quit) and then finally exit the game. Oh, and I have to confirm that I wish to exit, just in case I went through all that work by accident. Yes, I know this is probably only a PC issue and that the PC version suffers a lot more from not being properly adapted to keyboard and mouse, but it's still incredibly annoying. The keyboard and mouse problem can at least be solved, by plugging in a gamepad.

Conclusion

Don't get me wrong. I loved Assassin's Creed and I can't wait for the sequel, even though Philip Shahbaz will again be Altaïr's voice. I mean, let's give the guy a second chance, right? Seriously, though, I loved the game. Despite the obvious and perfectly avoidable problems, I recommend it warmly.

And if Ubisoft learns from their mistakes, the sequel should be even better.

Back in Blog

noreply@blogger.com (Anonymous) — Mon, 12 May 2008 18:05:00 +0000

Well, I think I finally got the hang of this being-a-dad thing; at least enough to get back to blogging. Suuuure, blame it all on the baby, that's convenient. Heh.

Anyway, things have been hectic in my life, but I'm back and I have stuff to rant about again. That said, I'm probably going to blog less code and include more variety. Oh and the posts are probably going to be shorter. Like this one. Not everything has to be a great story or an essay, right?

.NUTS: Yield Not, Waste Not

noreply@blogger.com (Anonymous) — Tue, 06 Nov 2007 22:40:00 +0000

Everyone agrees that iterators are cool. They're so cool, they've got their own design pattern. Even cooler than that, they made the folks at Sun introduce some (gasp!) new syntax into Java. Not to be outdone, C# goes even further: not only can you use iterators easily, you can also roll your own generators using the yield keyword. How cool is that?

To Litter or Not to Litter
As it turns out, it's not as cool as it sounds. If you care about memory allocation and garbage, then using yield is a bad idea. Before I go any further, let's explore this concern itself. Why would anyone care about memory allocation in a language with a garbage collector? Isn't the whole purpose of the garbage collector to abstract away memory allocation and help avoid all those nasty pointer-related bugs? If you find yourself nodding along to these questions, then I suggest that you read Joel's Law of Leaky Abstractions.

The reasons to care about garbage are diverse. In my case, it's because I'm working with XNA. That means that if I make too much garbage at the wrong time, the garbage collector will fire up and mess up my frame rate. As Shawn Hargreaves explained, there are two ways to prevent this. Myself, I prefer what he calls the Right Frequency Path, which is why using yield is a bad idea for me.

There are other kinds of software, where garbage might not be as critical an issue as in games, but you should still be aware of it. Too much garbage can cripple the performance of any system.

Garbage Generators

So far I have only claimed that yield is wasteful. Let's take a look at why it's so. Consider this short program:

using System;
using System.Collections.Generic;
using System.Text;

namespace Sandbox
{
 static class IteratorTest
 {
     public static IEnumerable<int> FibonacciYield(int max)
     {
         if (max > 1)
         {
             int a = 1;
             int b = 1;

             yield return a;
             yield return b;

             int c = a + b;

             while (c < max)
             {
                 yield return c;
                 a = b;
                 b = c;
                 c = a + b;
             }
         }
     }
 }

 class Program
 {
     static void Main(string[] args)
     {
         int sum = 0;
         for (int l = 0; l < 100000; l++)
         {
             foreach (int i in IteratorTest.FibonacciYield(17))
             {
                 sum += i;
             }
         }
         Console.WriteLine(sum);
     }
 }
}

Running CLR Profiler on this program might surprise you. Although there is no explicit allocation in the code, plenty of memory winds up in the landfill. The culprit is a class with a weird name: <FibonacciYield>d__0. In this simple example, it is responsible for a whopping 3.4 MB garbage pile. Where did that come from?

That's the price one has to pay for the magic. After all, when you write your own generator with yield, you're returning IEnumerable<T>, without even instancing it, let alone implementing it. But the return value has to come from somewhere and that somewhere is a compiler-generated class. If you run Reflector on the program and then take a look at the method FibonacciYield, you'll get this:

public static IEnumerable<int> FibonacciYield(int max)
{
  <FibonacciYield>d__0 d__ = new <FibonacciYield>d__0(-2);
  d__.<>3__max = max;
  return d__;
}

Just as suspected, that's where and how allocation happens. But what about our mysterious compiler-generated class with an ugly name?

Under The Hood

The dissasembly of the <FibonacciYield>d__0 class shows not only an implementation of IEnumerable<int> interface, but also a full-fledged IEnumerator<int> implementation. It has all the stuff a generator should have: the Current property, the Reset method and the MoveNext method.

At first, the code might look intimidating -- all this came out of that short method?! -- but a closer look reveals certain patterns. For example, take a look at the fields:

private int <>1__state;
private int <>2__current;
public int <>3__max;
public int <a>5__1;
public int <b>5__2;
public int <c>5__3;
public int max;

The <1>__state and <2>__current fields have to do with how the yield keyword itself works and I'll get back to them in a few moments. The rest show an interesting pattern: the local variables in the original method turn into <a>5__1, <b>5__2 and <c>5__3 fields, whereas the max argument becomes <>3__max field. But what about the max field? Why have both max and <>3__max? And why convert local variables into fields in the first place?

To answer that, remember how the foreach loop works: by calling the MoveNext method of the IEnumerator<T> interface and inspecting its Current property, until MoveNext returns false. This means that the state of whatever you're doing in your generator has to be preserved between two calls to MoveNext. In our case, the local variables of the original method are a part of the state that has to be preserved.

Speaking of state, the magic is explained when you look at the implementation of the MoveNext method:

private bool MoveNext()
{
   switch (this.<>1__state)
   {
       case 0:
           this.<>1__state = -1;
           if (this.max <= 1)
           {
               break;
           }
           this.<a>5__1 = 1;
           this.<b>5__2 = 1;
           this.<>2__current = this.<a>5__1;
           this.<>1__state = 1;
           return true;

       case 1:
           this.<>1__state = -1;
           this.<>2__current = this.<b>5__2;
           this.<>1__state = 2;
           return true;

       case 2:
           this.<>1__state = -1;
           this.<c>5__3 = this.<a>5__1 + this.<b>5__2;
           while (this.<c>5__3 < this.max)
           {
               this.<>2__current = this.<c>5__3;
               this.<>1__state = 3;
               return true;
           Label_00C5:
               this.<>1__state = -1;
               this.<a>5__1 = this.<b>5__2;
               this.<b>5__2 = this.<c>5__3;
               this.<c>5__3 = this.<a>5__1 + this.<b>5__2;
           }
           break;

       case 3:
           goto Label_00C5;
   }
   return false;
}

This is the heart of the compiler-generated finite state machine designed to react correctly to subsequent calls to MoveNext. The initial state is 0. Every time the compiler encounters a yield return statement, it creates a new state and generates the code that:

sets <>1__state to the new state
sets <>2__current to the value in yield return
returns true.

By the way, don't try to plug this code back into the sample program. The C# compiler will reject it, for reasons about which I will rant in another post.

The final mystery is resolved by looking at the GetEnumerator method implementation:

IEnumerator<int> IEnumerable<int>.GetEnumerator()
{
   IteratorTest.<FibonacciYield>d__0 d__;
   if (Interlocked.CompareExchange(ref this.<>1__state, 0, -2) == -2)
   {
       d__ = this;
   }
   else
   {
       d__ = new IteratorTest.<FibonacciYield>d__0(0);
   }
   d__.max = this.<>3__max;
   return d__;
}

After doing some tedious deciphering, this turns out to mean the following: if MoveNext That has already been called on this particular object, then create a fresh clone; if not, return this object. That's why we have both <>3__max and max: since arguments can be used as local variables within a method, the former is the original value supplied to FibonacciYield and the latter serves as a "local variable" field.

Making It Tidy

Now that we know how yield works, we can concentrate on avoiding memory allocation. Fortunately, C# allows you to write generators without implementing the IEnumerator<T> interface, as documented in help file:

In C#, it is not strictly necessary for a collection class to inherit from IEnumerable and IEnumerator in order to be compatible with foreach; as long as the class has the required GetEnumerator, MoveNext, Reset, and Current members, it will work with foreach. Omitting the interfaces has the advantage of allowing you to define the return type of Current to be more specific than object, thereby providing type-safety.

Omitting the interfaces, in this case, has the added advantage of being able to implement a generator as a struct, without it getting boxed later. Your final generator would look like this:

public struct ScroogeGenerator
{
    private int state;
    private int current;
    private int max;
    private int a;
    private int b;
    private int c;

    public ScroogeGenerator(int max)
    {
        this.state = 0;
        this.max = max;
        current = a = b = c = 0;
    }

    public int Current
    {
        get { return current; }
    }

    public bool MoveNext()
    {
        switch (state)
        {
            case 0:
                if (max <= 1) break;
                a = 1;
                b = 1;
                state = 1;
                current = a;
                return true;
            case 1:
                state = 2;
                current = b;
                return true;
            case 2:
                c = a + b;
                if (c >= max) break;
                a = b;
                b = c;
                current = c;
                return true;
        }
        return false;
    }

    public void Reset()
    {
        throw new Exception("The method or operation is not implemented.");
    }

    public ScroogeGenerator GetEnumerator() { return this; }
}

A few things to keep in mind:

When you write your own value-type generator, make sure you never cast it into an interface, or it will get boxed and make garbage. The best way to ensure that is not to implement any of the IEnumerable or IEnumerator interfaces.
The C# foreach loop does not call the Reset method. You can leave it unimplemented or empty if you'll use your generator in C# foreach loops only.
Don't reuse an instance of a value-type generator across different foreach loops. If you intend to do that, implement Reset and call it manually before reusing the generator.

.NUTS: Cemented Collections

noreply@blogger.com (Anonymous) — Wed, 19 Sep 2007 23:34:00 +0000

Another month has passed. My intention was to write another post sooner -- especially because I already had material for it -- but time flies when one is immersed in work. Now that I have some free time, I can sit down and share another one of my .NET anecdotes. And so, without further ado, welcome back to my .NUTS column.

Deja Vu

If there's one thing that defines a modern language as much as its syntax, it's the way it handles collections. Therefore, it shouldn't be such a big surprise that most of my encounters with .NET quirks stem from the differences between its collection classes and those I've used in Java. Still, I hope this is won't turn out to be a tradition, otherwise I will have to rename this column to "Collection Corner".

This particular tale, just like the last one, starts with the Dictionary generic class. Specifically, it starts with my need to be informed when someone manipulates its contents. The classes I was writing had a rather simple purpose: associate event handlers with different input controller events. One class managed keyboard events, another mouse events and yet another gamepad events. Each one of these classes exposed at least one read-only property of appropriate Dictionary type. All went smoothly until I decided that I needed to know when someone added, removed or changed the handler for an event.

Keep Me Posted

The first logical thing to do was to check the documentation and see whether the Dictionary class itself offered the desired functionality. Despite my years of experience with Java collection framework, I hoped Microsoft would surprise me pleasantly. Good thing that I didn't bet on it, though. After all these years of evolution, the two most sophisticated mainstream languages still don't offer the fundamental courtesy provided by Delphi's good old TStringList class.

Just like I expected, I decided I would have to implement the desired behavior myself. The logical choice seemed to be to override whatever manipulates the collections contents: the Add, Remove and Clear methods, and the indexer. If you have any .NET experience, you're probably chuckling at my naïveté by now; if you don't, then I'll keep you in suspense a bit longer.

Virtually Polymorphic

Perhaps the greatest difference between Java and .NET, certainly the one I find most difficult to get used to, is the way they handle inheritance polymorphism. In Java, all non-private methods are virtual by default. If you don't want the derived classes to override a method, you have to explicitly declare it as final. Once a method is declared as final, your derived classes cannot contain a method with the same name.

In .NET, all methods are non-virtual by default. If you want a method to be overridable, you have to explicitly declare it as virtual. If you're overriding a method, you have to include the override keyword. If you want to keep a virtual method from being overridden, you have to declare it as sealed. And if you want to introduce a new method with the same name in the derived class, you have to use the new keyword.

At first, this seems like a lot of fuss to achieve virtually the same results, pardon the pun. However, these differences are extremely important, as they represent different design philosophies. The .NET way makes you think twice before you decide which parts of your code will be flexible and which set in stone. As Eric Lippert will tell you, this mitigates the notorious Fragile Base Class problem, which should be cool, right?

No Jack For Plug

The aforementioned philosophical differences are not all about how many polymorphic angels can dance on the head of a non-private pin. There are some very practical consequences of making all your methods non-virtual by default. One of these consequences is rather unfortunate: none of the methods in the Dictionary class are virtual. In other words, there's no way to modify the behavior of the Dictionary class so that you get a notification if its contents change. Or is there?

Everyone is really excited about the shiny new features in C# 3, such as partial methods. It will be interesting to see whether the collection classes will benefit from this change. I haven't downloaded Orcas yet, so I don't know. I won't hold my breath, though. Besides, a new version of the language and the surrounding framework won't help those who need to extend the collection classes now. What solution could they use?

Interface/Off

Fortunately, there is a way to hack around both the language limitations and the rigid design of the collection classes, without too much work. The key is to fall back to the interface polymorphism. Where before I exposed my dictionary as a read-only property of generic Dictionary type, now I have to change its type to generic IDictionary interface.

The second step is to include the IDictionary interface in the list of base types for my dictionary class, like this:

class MyDictionary<TKey, TValue> : Dictionary<TKey, TValue>, IDictionary<TKey, TValue>
{
// ...
}

How does this help? It makes sure that the compiler will remap the interface methods to the corresponding methods in the class. If there is a method in MyDictionary whose signature corresponds to an IDictionary method, it will be mapped to it; if not, an inherited method will be used.

The final step is to declare the methods you want to modify as new methods:

public new void Add(TKey key, TValue value)
{
   // ...
   base.Add(key, value);
   // ...
}

That's all there is to it, really. Some things haven't changed that much since the bad old times of COM and COM+.

.NUTS: Enum Conundrum

noreply@blogger.com (Anonymous) — Mon, 13 Aug 2007 15:13:00 +0000

Welcome to .NUTS

I've been avoiding .NET for quite a long time. It's not that I'm an anti-Microsoft zealot, it's just that I used to code in Java at work and I figured that one platform full of frustrating limitations was more than enough. Recently, however, I decided to learn XNA on my own free time and that means getting involved with .NET and C#. It turned out to be just as frustrating as I feared, but at least it provided me with some nice blog material. Thus is the .NUTS "column" born in my blog, to document all that is weird in the .NET world. Expect stuff that might make you giggle, goggle and groan.

Enums Revisited

I first encountered the concept of enumerated types when I learned Pascal. Before that I owned a Commodore, so I dabbled in Simons' BASIC and 6502 assembler. Therefore, I didn't approach enums the way a C programmer might -- as a cleaner way of defining constants -- but as a way of defining a completely new type that accepts only those identifiers I define. The difference is subtle, but important: it shapes your attitude.

In .NET, enums have, obviously, inherited a lot of baggage from COM, which, in turn, comes from C. Furthermore, that baggage has been squeezed to fit the .NET luggage rack, where everything is an object, even the value types. Of course, value types are not really first-class objects, because they have to be boxed first. All in all, .NET enums are a part of a very leaky abstraction.

Starting Point

My journey of discovery started with an innocent little detail inside an excellent post on Shawn Hargreaves' blog. If you haven't read it yet, I strongly recommend it. I found it quite enlightening, especially the following advice:

If you use an enum type as a dictionary key, internal dictionary operations will cause boxing. You can avoid this by using integer keys, and casting your enum values to ints before adding them to the dictionary.

I found it strange that ints don't get boxed, while enums do. However, I was in a hurry to cobble my prototype together, so I just wrote it off as another one of those language eccentricities one encounters occasionally.

A couple of weeks later, I mentioned this to a friend of mine, who has been working in .NET since it first came out. He was even more flabbergasted: "That's weird. I mean, Microsoft keeps harping on how generic collections allow you to avoid unnecessary boxing. Seeing as how enum is one of the most common types for dictionary keys, the whole thing doesn't make sense at all."

That got me even more intrigued, enough to overcome my laziness and make me start poking around. This is where things started getting really interesting.

Now, those of you who don't care about the why and want the solution, skip to the part called "Finish Line", down below, near the end of this post.

Still reading? Okay, let's roll up those sleeves...

Stage 1: Observation

The obvious first step was to confirm Shawn's claim and work from there. For that, I armed myself with CLR Profiler (thanks to Mike Stall's blog) and I hammered out the following piece of code:

using System;
using System.Collections.Generic;
using System.Text;

namespace Sandbox
{
    enum TestEnum
    {
        e10,
        e9,
        e8,
        e7,
        e6,
        e5,
        e4,
        e3,
        e2,
        e1
    }

    class Program
    {
        static void Main(string[] args)
        {
            Dictionary<TestEnum, int> dict = new Dictionary<TestEnum, int>();

            for (int l = 0; l < 100000; l++)
            {
                TestEnum x = (TestEnum) (l % 10);
                dict[x] = 100000 - (int) x;
            }

            for (TestEnum x = TestEnum.e10; x <= TestEnum.e1; x++)
            {
                Console.WriteLine(dict[x]);
            }
        }
    }
}

The results were as bad as Shawn predicted: the code allocated 4,825,246 bytes (more than 4.5 MB), out of which 74.61% were instances of Sandbox.TestEnum type. If an enum is winding up on the heap like that, it must be getting boxed.

The next thing I did was to change the Main method to look like this:

        static void Main(string[] args)
        {
            Dictionary<int, int> dict = new Dictionary<int, int>();

            for (int l = 0; l < 100000; l++)
            {
                int x = l % 10;
                dict[x] = 100000 - (int) x;
            }

            for (int x = 0; x < 10; x++)
            {
                Console.WriteLine(dict[x]);
            }
        }

The profiler reported 24,692 allocated bytes, with nothing sticking out as especially wasteful. It was clear that .NET handled ints and enums differently, even though they are both value types.

CLR Profiler has a very neat feature that allows you to find out which part(s) of the code allocated the memory, by right-clicking on the offending bar in the histogram view and selecting the "Show Who Allocated" option. That will open the allocation graph view, which in my enum case looked like this:

(click on the image to enlarge)

Apparently, my enum was being boxed by ObjectEqualityComparer, inside the Insert method in the generic Dictionary class. Culprit identified, time to move on to the next stage.

Stage 2: Under the Hood

Contrary to my first impulse -- whip out the IL Disassembler -- I decided to first check whether the help files mentioned ObjectEqualityComparer. No such luck. ILDASM it is, then. The disassembly of Insert method in the generic Dictionary class revealed the following interesting snippet:

  IL_001e:  ldfld      class System.Collections.Generic.IEqualityComparer`1<!0> class System.Collections.Generic.Dictionary`2<!TKey,!TValue>::comparer
  IL_0023:  ldarg.1
  IL_0024:  callvirt   instance int32 class System.Collections.Generic.IEqualityComparer`1<!TKey>::GetHashCode(!0)

So my suspect, ObjectEqualityComparer, is an implementation of an IEqualityComparer interface, contained in the comparer field inside my Dictionary object. Here's what help file says about IEqualityComparer:

This interface allows the implementation of customized equality comparison for collections. That is, you can create your own definition of equality for type T, and specify that this definition be used with a collection type that accepts the IEqualityComparer generic interface. In the .NET Framework, constructors of the Dictionary generic collection type accept this interface.

A default implementation of this interface is provided by the Default property of the EqualityComparer generic class. The StringComparer class implements IEqualityComparer of type String.

A bit more digging in help files revealed this:

Dictionary requires an equality implementation to determine whether keys are equal. You can specify an implementation of the IEqualityComparer generic interface by using a constructor that accepts a comparer parameter; if you do not specify an implementation, the default generic equality comparer EqualityComparer.Default is used. If type TKey implements the System.IEquatable generic interface, the default equality comparer uses that implementation.

So how does the EqualityComparer.Default get defined? According to the help file, like this:

The Default property checks whether type T implements the System.IEquatable generic interface and if so returns an EqualityComparer that uses that implementation. Otherwise it returns an EqualityComparer that uses the overrides of Object.Equals and Object.GetHashCode provided by T.

In other words, the boxing happens because my enum doesn't implement IEquatable interface. The funny thing is that Int32 value type implements IEquatable, as does every other primitive numeric type. Yet enums don't. When you stop to think of it, it sounds logical: the Enum value type can't implement IEquatable because that would theoretically allow you to compare any enum to any other enum and obtain bogus results. Not that I'm convinced by that argument, but it's really a moot point, seeing as how I can't change that myself. That leaves looking for a workaround, which brings us to the last stage.

Stage 3: The Lesser Evil

Once you're reduced to looking for a workaround, your job is to look for the least dirty and the least ugly solution. The first thing I tried was to make my enum implement the IEquatable interface. Naturally, there was no way to make C# swallow that syntax. Next I tried setting the EqualityComparer<TestEnum>.Default property, but unfortunately it's read-only. Finally, grudgingly, I conceded to write my own implementation of IEqualityComparer. The first attempt was to write it as a generic class, but when C# complained about not being able to explicitly cast an unknown value type to int (within my GetHashCode method) it finally drove the lesson home: .NET generics are definitely not the same thing as C++ templates.

The final solution looked like this:

    class TestEnumComparer : IEqualityComparer<TestEnum>
    {
        public static readonly TestEnumComparer Instance = new TestEnumComparer();

        #region IEqualityComparer<TestEnum> Members

        public bool Equals(TestEnum x, TestEnum y)
        {
            return (x == y);
        }

        public int GetHashCode(TestEnum obj)
        {
            return (int) obj;
        }

        #endregion
    }

The dictionary declaration would then look like this:

            Dictionary<TestEnum, int> dict = new Dictionary<TestEnum, int>(TestEnumComparer.Instance);

Finish Line

To sum it up, for each enum type you intend to use as a dictionary key, you'll have to write your implementation of IEqualityComparer and pass it to Dictionary constructor. For each enum type you'll have to type 18 lines of boilerplate code -- you can squeeze it into less, I know -- but that's a relatively small price to pay for avoiding the memory waste caused by boxing your keys. I just wish it wasn't that ugly, but at least it's simple enough. Stay tuned for more .NUTS in the future.

Dark Side of Design Patterns

noreply@blogger.com (Anonymous) — Fri, 01 Jun 2007 20:10:00 +0000

Every now and then I overhear someone excitedly talking to someone else about Design Patterns. They're so excited about it, you can hear the capital D and capital P quite clearly. And they use that tone: you know, the one usually reserved for great sports achievements. And, invariably, I can't resist the temptation to get involved. Things usually go downhill from there.

I can understand their chagrin. How would you like it if you just discovered something cool only to have some spoilsport tell you that roughly half of it is crap? Yet I just can't keep quiet; partly because I'm mean and love seeing their reactions, but mostly because I'm sick of the hype.

Patternitis

"The whole problem with the world is that fools and fanatics are always so certain of themselves, but wiser people so full of doubts."

Bertrand Russell

Don't get me wrong, I think design patterns are a great idea. Hardly a new one, but great nevertheless. The problem is that every Great Idea attracts its share of fanatics who think it's the coolest thing since sliced bread. It gets paraded around as a Silver Bullet and a Golden Hammer until its limitations percolate through the layers of hype. Inevitably, the general hype dies down -- if we're talking about a genuine Great Idea, not glossyware stuff like SOA -- and all that's left are pockets of hype that form about enthusiastic newbies or incompetent know-it-alls.

I personally love stamping out those little pockets and that's what I want to do with this blog post. I'll start by asserting the following:

If you think that developing software can be reduced to applying and combining design patterns, you are wrong.
If you think that a design pattern you don't completely understand must nevertheless be good, you are wrong.
If you think that design patterns are a new invention made possible by OOP, you are wrong.
If you think that someone is a good programmer just because they know design patterns, you are wrong.
If you think that you must know design patterns to be a good programmer, you are wrong.

Now that the most common misconceptions are out of our way, we can get to separating the wheat from the chaff.

Gold vs. Other Glittery Stuff

"Any man whose errors take ten years to correct is quite a man."
J. Robert Oppenheimer

What is a design pattern? The original book by Gamma, Helm, Johnson and Vlissides (popularly known as GoF) takes a page and a half to answer this question. I prefer a shorter and simpler answer, even if it lacks all the accuracy of the original. Wikipedia has pretty good definition:

In software engineering (or computer science), a design pattern is a general repeatable solution to a commonly occurring problem in software design. A design pattern is not a finished design that can be transformed directly into code. It is a description or template for how to solve a problem that can be used in many different situations. Object-oriented design patterns typically show relationships and interactions between classes or objects, without specifying the final application classes or objects that are involved. Algorithms are not thought of as design patterns, since they solve computational problems rather than design problems.

Still a bit long for my taste, but it's quite usable. As a matter of fact, the first sentence is the most important; the rest just helps cement the scope.

The GoF book defines 23 design patterns: 5 creational, 7 structural and 11 behavioral. It took me a lot of conscious effort and quite a few years of working experience to admit that not all of those patterns are gold. In fact, the wheat-to-chaff ratio is quite close to 50 percent: there are 12 that I could describe as "okay", "good" or "great" and 11 that I see as "hack", "poor" or even "rubbish".

The Good

"The remarkable thing about Shakespeare is that he really is very good, in spite of all the people who say he is very good."
Robert Graves

If I learned something from my previous managers, it's to always dangle the carrot before using the stick. Seriously, though, the original GoF book has some really good patterns in it.

Take Bridge, for example. The best thing about this pattern is that it teaches you about the separation of concerns. You can adapt it to classless object-oriented languages, you can even apply it in structural programming languages. The main point is to keep the implementation primitives separate from the operations that use them.

Builder is another example of a highly useful pattern. Just look at SAX: they provide the director and you have to write the builder. Highly efficient and very elegant. By the way, I really recommend reading the original GoF book if you want to understand this pattern. Some of the other books attempt to explain it by applying it to convoluted problems such as constructing a MIME message or building a user interface. A word of advice to the authors: KISS.

A few other great patterns include Command, Composite and Template Method. These really make the best out of OOP and it shows. These are all nice examples of taking advantage of the benefits of OOP.

Then there is a whole range of patterns that have been around for a long time, long before OOP and long before GoF invented design patterns as we know them: Adapter, Façade, Flyweight, Chain of Responsibility and Observer.

Finally, I reserve a special place for Mediator, not because it's better than the rest, but because it's a perfect example of a good pattern that can be horribly abused. One word: Delphi. The "RAD paradigm" promoted by Delphi is to add objects to their containers and then write all the logic in a few huge Mediator classes. Delphi is based on an object-oriented language, has an excellent core framework and class library and yet it teaches its users to forget OOP. Bottom line: no matter how cool the pattern, don't overdo it.

The Ugly

"Beauty is the first test; there is no permanent place in the world for ugly mathematics."
G.H. Hardy

We have all, at some point in our lives, had to choose the lesser of two evils. That's most likely what you will be doing when you decide to apply one of the following: Proxy, Memento, Visitor, Decorator or State. All of these are a cumbersome solution for something that should have been supported or aided by the language itself.

Proxy, for example, is a necessary and useful pattern. The problem with it is that a lot of the languages require you to implement it by writing tons of boilerplate code. With some good metaprogramming facilities, Proxy becomes trivial.

Likewise, the motivation behind Memento is valid, but it should be solved by the language or its core library. Serializing and deserializing objects should be trivial, which means you shouldn't have to write gobs of code to do it. Look at the way you do it in Java:

ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(oos);
try {
  oos.writeObject(originator);
  oos.close();
} catch (IOException e) {
  // won't happen but we _still_ 
  // have to write code to ignore it
}
byte[] memento = baos.toByteArray();

Now compare it to Ruby:

memento = Marshal.dump(originator)

I rest my case.

Visitor is another example of hacking your way around language limitations. Most of the object-oriented code anyone writes needs only single dispatch. You can argue that supporting double or multiple dispatch in the language is wasteful, although there are ways around that. But when your language leaves you with no other choice and you need double dispatch, you will have to use Visitor. If you happen to need multiple dispatch, good luck writing and maintaining that code.

Finally, we come to Decorator and State. Both of these patterns are useful and both deal with the same issue: sometimes we want to adapt the behavior of individual objects depending on their state or some other trait we want to manipulate during run time. I believe that it should be possible without writing all the boilerplate code that you need for Decorator and State. I've learned and experimented with Self and Io languages and they both allow you to solve these needs very elegantly. Again, look at a sample implementation of State pattern in Java:

public class Bot {
  
  private int health;
  private BotState state;
  
  protected void changeState(BotState newState) {
    state = newState;
  }
  
  protected static final BotState HEALTHY = new HealthyState();
  protected static final BotState INJURED = new InjuredState();
  
  protected abstract static class BotState {
    public void takeDamage(Bot bot, int damage) {
      bot.health -= damage;
    }
    
    public void heal(Bot bot, int delta) {
      bot.health += delta;
    }
    
    public void enemySpotted(Bot bot, Object enemy) {}
  }
  
  protected static class HealthyState extends BotState {
    public void takeDamage(Bot bot, int damage) {
      super.takeDamage(bot, damage);
      if (bot.health <= 40) {
        bot.changeState(INJURED);
      }
    }
    
    public void enemySpotted(Bot bot, Object enemy) {
      bot.chase(enemy);
    }
  }
  
  protected static class InjuredState extends BotState {
    public void heal(Bot bot, int delta) {
      super.heal(bot, delta);
      if (bot.health >= 50) {
        bot.changeState(HEALTHY);
      }
    }
    
    public void enemySpotted(Bot bot, Object enemy) {
      bot.avoid(enemy);
    }
  }
  
  public void chase(Object enemy) {
    System.out.print("Chasing " + enemy);
  }
  
  public void avoid(Object enemy) {
    System.out.print("Avoiding " + enemy);
  }
  
  public void takeDamage(int damage) {
    state.takeDamage(this, damage);
  }
  
  public void heal(int delta) {
    state.heal(this, delta);
  }
  
  public void enemySpotted(Object enemy) {
    state.enemySpotted(this, enemy);
  }
}

Now compare it with the same thing in Io:

Bot := Object clone do (
  init := method(self health := 100)
  chase := method(enemy, ("Chasing " .. enemy) println)
  avoid := method(enemy, ("Avoiding " .. enemy) println)
  takeDamage := method(damage, health = health - damage)
  heal := method(delta, health = health + delta)
  enemySpotted := method(enemy, nil)
  changeState := method(newState, self setProto(newState))
)

HealthyBot := Bot cloneWithoutInit do (
  takeDamage := method(damage,
    resend
    if (health <= 40, changeState(InjuredBot))
  )
  enemySpotted := method(enemy, chase(enemy))
)

InjuredBot := Bot cloneWithoutInit do (
  heal := method(delta,
    resend
    if (health >= 50, changeState(HealthyBot))
  )
  enemySpotted := method(enemy, avoid(enemy))
)

I would even settle for hard, specific syntax supported by the language itself, like in UnrealScript.

Bottom line? Our languages really need to evolve. Combine prototype-based OOP that you can find in Self and Io with excellent metaprogramming offered by Lisp and you will have a language that kicks ass and doesn't need ugly patterns.

The Bad

"There is no stigma attached to recognizing a bad decision in time to install a better one."
Laurence J. Peter

Thus we come to the place where the bad things dwell. There's a variety of patterns that fall into this group and they do so for diverse reasons. Let's start with a couple that gave me nightmares for a long while.

Factory Method sounds like a good idea. Upon close examination, Abstract Factory is revealed to be Factory Method applied to Factory Method, which makes it sound even better. Only those who have been fooled by this pattern can appreciate the despair and the subsequent numbness brought on by writing tons and tons of the boilerplate code required by it. The worst thing is that it seems to be the only solution. Until you find out about Dependency Injection. Try it and you'll see the difference.

Another pattern that really fouls things up is Singleton. For a long time I couldn't find one good reason why I should have a class with only one instance, instead of making all that code static. Then someone asked what I would do if I had to pass an interface to it. Okay, that justifies it, I can even think of examples other than boilerplate Abstract Factory code. Another valid point is that you might want to switch from one instance to a pool and you don't want to rewrite the bulk of your code. Then there is the case mentioned in the GoF book that talks about subclassing the singleton and creating a registry of singletons. If you think that things are getting out of hand, you're right. The solution is the same as above: use Dependency Injection.

Then there's Iterator pattern. That one should be easy: use the foreach statement. And if your language doesn't have it, ask for your time back. It's simply unacceptable to have to write reams of ugly code for something we use so often. It deserves its own syntax.

What about Strategy pattern? According to the GoF and to all subsequent authors, you should use it to encapsulate a family of algorithms and make them interchangeable. That's about one thing for which I would not use it. If you want to do that use functions. Again, if you don't have them, ask for your time back. On the other hand, you could use Strategy pattern when your implementation strategy has to maintain state over time. But then it's not really Strategy, is it? Slippery things, these patterns.

Finally, there's Interpreter. I don't have any beef with its motivation: just look at how things have been evolving and you'll see several examples of minilanguages, scripting languages and XML-based stuff, all getting interpreted in the normal course of execution of your code. From printf format string of olden to XML configuration files of today, the idea has been in use for a loooong time. Thing is, this isn't a design pattern. It doesn't fit into the scope of a design pattern. And you won't make it fit by sketching out your favorite implementation of an interpreter for your favorite minilanguage. It just does not belong.

Patternitis Revisited

"When people are free to do as they please, they usually imitate each other."
Eric Hoffer

I have to give it to GoF though, they had some great ideas and the only time their scope slipped was with the Interpreter pattern. Sadly, the same couldn't be said about the later authors. I'll say it out loud: interface is not a design pattern. There are many things you could say about an interface. From a conceptual point of view, it's the definition of the contract your code will have to satisfy. From a language-specific point of view, it's a syntactic and semantic construct. But it is definitely not a design pattern. Neither is a symbolic constant name or an accessor method name.

It might seem that I'm beating a dead horse here, but I want to make it absolutely clear that not everything in software development is a design pattern. It's too easy to lose focus in enthusiasm, so do yourself a favor: learn the design patterns, keep using the ones you like (where applicable) and move on. There's a lot more to do.

Uneducational Crisis Pt. 3: Approach

noreply@blogger.com (Anonymous) — Tue, 06 Feb 2007 14:16:00 +0000

Most of the things I wrote in the previous two posts can be disputed with counter-examples: not every professor is a Prima Donna; not all things they teach you at the university are irrelevant; there are great places where smart and likable people teach you good stuff; everything else is just "bad luck" and you "have to take the good with the bad". Let me tell you what I think about that:

I don't buy it.

There's a big problem with education as a system in general, not just in the CS field. And that problem involves the relevance of what we learn, from kindergarten to university. How many things did you learn in the ground school that you have completely forgotten? Let's face it, as a society we are doing a shoddy work of teaching our kids.

Why is education so bad? There is probably a variety of factors that enter the whole equation, but I think there are only two of major importance. One of them is the fact that teaching is not the number one motivation behind the existence of schools. Paul Graham has put it succinctly in his essay "Why Nerds are Unpopular":

Teenagers now are useless, except as cheap labor in industries like fast food, which evolved to exploit precisely this fact. In almost any other kind of work, they'd be a net loss. But they're also too young to be left unsupervised. Someone has to watch over them, and the most efficient way to do this is to collect them together in one place. Then a few adults can watch all of them.

The other major factor is that the basic approach to teaching is wrong. And that's what I want to explain here, so let's delve into it.

What Do You Want to Be When You Grow Up?

Whenever someone asks me how I came to be so obsessed with computers and programming, I tell them to blame my dad. No, he didn't teach me how to code, nor did he teach me anything important about the computers beyond turning them on and off and washing my hands before touching the keyboard. In fact, he can't code to save his life, which is perfectly fine, because he's a writer. What he did do, though, is much more important: he introduced me to computers.

How did he know I would like computers? That's the whole point: he didn't. When I was 5 or 6 years old -- I don't remember clearly anymore -- he started showing me the basics of photography. I found it entertaining, but I didn't take it too seriously. Then he tried with cinematography and the results were more or less the same. But then he showed me a computer and since that first look I am completely and totally fascinated with computers and the programs they run.

Why am I telling you this? Because I think I was extremely lucky. From that moment on, I knew what I wanted to be when I grow up; I had an interest that didn't pass and fade and I always had a new goal to achieve and a new way to improve myself.

I do know a few other people who have a passion in their life. I also know many more who never discovered anything they really like and lead largely disoriented lives. It sounds harsh, but it's true: having a spouse and kids cannot replace the feeling of having your own dreams. You can share your dreams with your family and complement one with the other, but in the end they are not interchangeable.

Nosce Te Ipsum

Here's the interesting part: I think everyone can and should have the chance to discover what they like. That's one of the most important things in our personal development and we usually don't have much help with it.

That's why most of the stuff they teach us at school is useless and irrelevant. How do you know something is relevant? You don't. You have to discover it. It's not something you can be taught, it's something you have to find out yourself. But that doesn't mean you shouldn't have help.

Educational institutions are ostensibly there to impart knowledge we will need in order to be useful and productive members of the society. But we don't know where and how we fit into that society. And the society doesn't know it either, so they just try to pour all sorts of stuff on our developing minds, hoping that after a while we'll get an idea where to head and how to specialize.

Optimism is good, but that's not optimistic: it's plain ridiculous. Imagine you had to pick what kind of food you want for your wedding party and the way to choose is to first learn by heart the recipes for a whole lot of different dishes and then use that to decide.

Yes, you will have an idea what each dish tastes like and yes, you might even make a dish or two to check out the taste. You might even get lucky and choose well. But if you made a mistake, there's no turning back: your wedding party will have food you don't like and that's it. You might get married again later and have another wedding party, but this one is gone for good. And the food sucked.

Of course, in real life you can get help from dedicated experts who specialize in that sort of stuff.
Imagine if you could get help with discovering your own talents and interests. Imagine if you could get that kind of help from people who specialize in it. Wouldn't life be a whole lot nicer?

Rat Races

Instead of that, what do we have now? We have to assimilate a considerable quantity of knowledge in a fixed time period, at the end of which our performance is rated and, if it is found satisfactory we can proceed to the next period and the next batch of knowledge.

There's a name you can slap on that description: competition. Our educational paradigm is based on competition and that makes it not merely misguided, but downright harmful. Instead of helping to find what our kids can do well, we force them to become "good enough" at doing a bit of everything. Worse, after passing through our educational maze, the kids are still not really good at doing anything of real worth.

Consider the most notable effect of our educational model: cheating. I really like how Wikipedia hit the nail on the head in their article:

A common venue for cheating is in education settings, where it takes a number of forms.

Of course it does. If you're competing for something that could bring you a gain you can clearly perceive, you might be tempted to cheat. But if you're forced to compete at something that you don't even perceive as lucrative (and nobody else really believes it either), you won't be tempted to cheat, you will consider it a natural option.

I'm not against the competition in education. Competition can be good, because it fosters a drive towards self-improvement. But you have to want to compete. And that happens either when you're competing at something you like or for something you perceive as lucrative.

Silver Bullet?

Of course, the solution is certainly a lot harder to implement than to propose. And I didn't even propose a complete solution, I just gave an idea. And I'm probably not the first to have it or even to voice it. And it certainly wouldn't solve all the problems and cure all the diseases and stop all the wars.

But don't you think it would be nice to try it?

Uneducational Crisis Pt. 2: Subject Matter

noreply@blogger.com (Anonymous) — Fri, 19 Jan 2007 12:41:00 +0000

When I think about these last four years, I don't usually think about the university. Much more important and interesting things have been happening in my life. I think that says something about the quality of education as a life experience: you only consider it important when you're forced to do it during the better part of your day.

But when I do think about the university, the first thing that comes to mind are the professors. I've had a few excellent professors who were a pleasure to study with, but the rest ranged from merely disinterested to downright inept. In my previous post, I covered this topic in detail.

The second thing that comes to mind is what I learned. Or rather, what I was supposed to learn. Now that the things are drawing to their unceremonious end, it's amazing to look back and see just how many things I've been told were

misleading
wrong
obsolete
stupid
some or all of the above

With a lot of real-life experience in software industry, you can afford to be told such things because your brain will reject them automatically after a brief examination. But the reason I started writing about the educational problems in CS is not because I feel cheated out of my money, but because of the kids who go to the university to try to learn.

You see, I love programming and I like doing it for a living. And I enjoy the company of other people who do it. Sure, school might not matter if you're good enough (and infatuated enough). The knowledge is out there and you can learn it all by yourself. But you might not find it soon enough and you might get the wrong ideas at school and you might come out of it all utterly confused or frustrated or suffer some other damage. There's enough stuff in real life that will confuse and frustrate you, no need to start out like that.

So what's wrong with the stuff they teach you at the university?

Obsolescence

I agree that learning about the past is important. You need some context if you want to understand not only the how, but also the why of the things you see today. But that's no excuse for teaching obsolete stuff.

Take waterfall model, for example. For a big enough project, as most of real-world projects are, it simply does not work -- there are too many things that can go wrong and will go wrong. And yet the universities insist on teaching it as the way to execute software development projects.

Another good example is the insistence on Design Flow Diagrams. I'm not implying that DFDs are obsolete. Nor am I saying the same about any other concept from SSADM. Had any of our professors bothered to teach us SSADM well, I might have learned enough to compare it to what I'm used to working with.

As far as DFDs go, I find them stunningly imprecise and uncomfortable, compared to UML. This, of course, is a topic for a discussion that goes beyond the scope of this post, but it's also a bit beside the point. The point is that if the course is called Systems Analysis and Design, I don't think it can stay locked in time forever, covering SSADM in depth and merely mentioning the existence of UML and iterative processes and the myriad of other techniques that evolved in the mean time.

Scope

The problem with SSADM vs. UML is just an specific case of a bigger and more general problem: the scope is all wrong. If you are going to give an OOP course, you should take care to explain what OOP is, what OOP isn't, why it's been invented, how it works, etc. Sounds reasonable, but it doesn't happen. What you get instead is a Java course, where in the first few classes you get a very brief, confused and inaccurate summary of OOP concepts and then you delve into the language specifics.

I love learning new languages. As Kevin Kelleher notes, each language tries to solve some problem. That means that each new language you learn might teach you about a new class of problems or a new class of solutions. The thing is, you have to realize what the problems are and why they are problems before you can benefit from learning a new language. Otherwise you're just learning syntax.

You could do it the way I did it: learn a new language, compare it with the one you knew before and gradually become aware of the new problems. After enough time and enough new languages, you'll be able to think of languages in terms of problems and solutions. Indeed, nothing can completely replace the process of learning from your own experience. But that doesn't mean you aren't entitled to decent formal education.

It's like your parents telling you not to play with fire. You might heed their advice and still get burned by something you didn't know was hot. Or you might disregard their advice and get burned. But if they didn't tell you not to play with fire you would surely get burned out of ignorance.

Specialization

The problem of scope is what you'll see on the individual course level. But when you take a step back and examine the entire course curriculum, you'll find yourself facing the opposite extreme: breadth. The sheer breadth of subjects covered combined with the scope of each course makes you wonder why you got yourself into this at all.

Here's a bit of news for the guys who design the educational program: most of the software developers don't need to know about the implementation specifics of the datalink layer in ISDN. Nor do most network experts really need to know how to design a database in the third normal form.

I understand why the universities do this. It's a lot more manageable to have your courses flow sequentially and give everyone who survives the same diploma. Teach them all and let job sort them out. The question is, should ease of management excuse the poor results? Let's put it to a test: what would you think of a manager who told you, before the project started, that you'll have to implement a web-based user interface, a Swing GUI and an interactive textual interface, because it's easier to plan the project that way than to plan for capturing the UI requirements?

Relevance

All of the problems I've described here could be easily summarised in one word: relevance. Obsolete subjects are irrelevant to what you'll be doing. Courses with wrong scope teach you irrelevant things. Lack of specialization ensures you'll be learning a lot of irrelevant stuff. I postulate that every problem with the subject matter in CS education can be filed under relevance.

So how do we solve this? I hate doing this, but I'm afraid I have to do a Robert Jordan and leave you hanging. By now I've built enough context to engage in a discussion that I wanted to start from the beginning, but that discussion merits its own post. Therefore, I have no choice but to say RAFO.

At least I promise to finish in the third post.

Uneducational Crisis Pt. 1: Human Factor

noreply@blogger.com (Anonymous) — Tue, 16 Jan 2007 17:37:00 +0000

If there's one thing worse than a bad programmer, it's a failed programmer turned professor. I don't know how common this is in United States, but here in Chile you see it a lot. And try as I might to pretend that I don't care, it's actually driving me nuts. Quite understandably, too: I could be doing interesting things instead of trying to guess what the latest in a row of brain-dead dilettantes wants me to write as a solution to his "ingenious" exam exercise.

On the other hand, this way I get the first-hand insight into the problems with education in CS field. Some of these problems, I suppose, are local and owe their existence to a variety of economic, political and technological factors, but I'm convinced that some are universal. Nonetheless, I shall not discriminate: I'll go over them all and a plague on both their houses.

This article, the first in the series, deals with the professors themselves.

Two Roads Diverged in a Yellow Wood...

Over the time, I learned to distinguish two types of CS professors:

The Geek. This guy knows his stuff, because he actually works in computer industry and does his job well. Sometimes he is motivated by a genuine desire to teach new generations, but often he is teaching just to make a bit of money on the side or to lend a bit of academic weight to his name. More often than not, he is wonderfully inept at imparting his knowledge to the students, because he's a computer geek, not a teacher. All in all, he's guaranteed to be relaxed, easy-going and flexible. Mostly harmless.
The Prima Donna. Who needs knowledge when you have Orwellian authority? This is the guy who will typically boast at least 20 years of experience in computer industry, who has a prominent consulting company and is certified by Carnegie Mello [sic] University. His classes are pure poetry. Vogon poetry, to be precise.

The Prima Donna is the type that does the most damage, so he merits detailed dissection.

Spot the Prima Donna

A Prima Donna professor is rather easy to recognize. He will exhibit a combination of several seamlessly coupled behavioral patterns from the following list:

Smugness. Usually one of the first signs that you're dealing with a Prima Donna is that self-satisfied attitude aimed to make you feel that some day, with lots of effort and a divine intervention or two, you'll get to be as good as this guy.
Hand Waving. One of the most valuable techniques for explaining the unknown is the hand waving. To those who came to learn, it seems that the Prima Donna is making a herculean effort to explain the concept and that they are simply too dumb to understand him.
Stating the Obvious. A good presentation slide will show only the key concepts and leave the explanation to the presenter. A bad presentation slide will show all the information in a pile of text you won't bother to read. In both cases, the Prima Donna will repeat the contents of the slide, with lots of Hand Waving, and you'll be left no wiser than before.
Exasperation. Sometimes a braver student will ask the Prima Donna to clarify something previously "explained" by Hand Waving and Stating the Obvious. If the Prima Donna is not in the mood to do it all again, he'll inform the audience that he explained it several times already.
Selective Hearing. If there's a student willing to press the issue despite the Exasperation, the Prima Donna might choose to answer another student's unrelated query or to simply go on to the next topic.
Condescension. You never know when you might get lucky: sometimes a Prima Donna will even give away unearned points when correcting an exam. But don't worry, he will make sure to inform you about it.
Objective Subjectivity. He might be teaching you how to solve problems that have more than one correct solution, but you better make sure you solve the problems the way Prima Donna taught you to do it.
Asserting the Authority. The Prima Donna will make sure you don't forget that he's the professor and you're the student, which means you owe him certain respect. He'll be especially sensitive about certain forms of disrespect, such as disagreeing with him.
Showcasing the Merits. In case you don't seem to be convinced of the Prima Donna's authority, you'll be informed of one or more of his certifications, nominations, recognitions, citations, publications, you-get-the-idea-tions.

Occasionally, you might get really unlucky and wind up with an embodiment of the archetypal Prima Donna. If there is an option to get the authorities to replace him, go for it.

Damage Done

It's becoming increasingly popular to be a nerd. Sure, you're still going to be bullied in school, but at least now you know you might be in store for a bright future. And those around you know it too. People tend to pay attention to things that might make them rich and when one of the richest persons in the world is downright nerdy, you can bet it won't go unnoticed.

The net result is that you get a lot of CS students who are in it for the glory and riches. These people don't know a lot about computers, but that's okay, because they're there to learn. The problem is that a lot of them also lack the talent. They're simply not good at computer science or any natural science at all.

Imagine all these people trying to learn about pointers from a Geek professor. Confusion abounds, chaos reigns supreme and in the end a lot of them will either fail the course (and hopefully change major) or pass the course having learned nothing.

Now imagine them trying to learn about pointers from a Prima Donna. Apart from becoming utterly frustrated and dejected, they'll learn a lot of garbage. The things that will stick the most are the parts that the Prima Donna invented or misinterpreted, because those are the most vivid explanations the Prima Donna will offer. That's the core of what makes him an "eminence" in the "computing world".

And now imagine yourself working with them on a project. Imagine working on a piece of code that has to traverse a tree with a co-worker who believes that "recursion is for programmers who have no style." Imagine hunting down an interoperability bug caused by bad handling of character encoding in a code written by a person who has been taught that "ASCII table is physically implemented in the microprocessor." Imagine writing business logic that has to use a database designed by someone convinced that a transactional database must satisfy the fifth normal form.

Not a cheerful thought, is it?