The CodeWrights Tale

SharePoint 2010 Custom IFilter development update

2011-12-29T14:43:00.000-06:00

The previous post on IFilter development for Windows and SharePoint, I included several links to resources we used to code a custom IFilter; the hope being they would be useful to anyone looking to write their own, seeing as how difficult it was to find accurate information on the subject.

Alex Culp did an outstanding job with the IFilter, so much so that Microsoft asked him to write a MSDN article on the subject of writing a custom IFilter. The custom file type wasn't the issue, it was all the complexities of C++ and IFilter development combined with registry entries, properties, 32bit/64bit details and so forth, like how the 64-bit version of Visual Studio creates a 32-bit installer by default which then didn't install the correct version of the .Net dlls. Another bit-ness nugget is to pay attention to which Regedit you run when setting registry entries; 64-bit entries are not visible to the 32-bit executable.

Mysteries of IFilter Development Revealed

2011-10-03T16:40:00.001-05:00

Alex Culp and I worked on a IFilter for a customer file type. His page on writing an ifilter is down while he works on publishing an article on writing ifilters for Microsoft. [Update: His MSDN article on IFilter development has been published, see my post, http://codewright.blogspot.com/2011/12/sharepoint-2010-custom-ifilter.html] In the meantime, I figured I'd share my list of IFilter development links. Considering all the other technology out there and the age of the ifilter interface, I was surprised how difficult it was to find useful information.

A good start is this MSDN page: http://msdn.microsoft.com/en-us/library/ms916793.aspx. It's somewhat old but does cover lots of the details involved. We had the hardest time finding out how to create a custom property or property set for the ifilter to use when it found metadata. We ended up using an existing category, Basic, and creating property ID's above the range of existing properties, i.e. we started at 200 since the existing Basic properties didn't get above 100.

Here's info on debugging IFilters in MOSS. We did have trouble getting all the registry entries needed to properly register the IFilter with the server. Keep a log of the entries, there are quite a few and in confusingly similiar places, especially since GUIDs are involved.

Here is another page on ifilter development. Keep reading through the comments, lots more details are discussed there.

IFilter explorer. Shows information about the installed ifilters.

Good article series from Anne Stenberg on crawled properties and how to find the one you want to map. This is useful in understanding more about a crawled property, like when you want to make your own.

This post mentions a dev guide to writing your own SP2007 protocol handler. Might be a good resource.

Computer Science is an Art

2011-01-11T09:00:00.002-06:00

Of the many eternal debates that spiral around software development blogs the one that seems the most intransigent is whether the practice of programming is an art or a science.

Chefs have a similar discussion over cooking and baking. Cooking is an art while baking is a science. Why? Because when you put something in the oven, you had better get it right the first time. Mistakes mean starting over. Cooking, on the other hand, allows the chef the leeway to make changes as the dish progresses. The chef can take action to keep the dish from heading off course. For example, when making gravy you keep adding flour until it is the right consistency and if you add too much flour you can add some milk to thin it down. It is incremental by nature.

Computer Science or Software Engineering - An Elemental Problem

2010-12-01T09:20:00.002-06:00

What is the difference between Computer Science and Software Engineering? That depends on the nature of the problem you are trying to solve - is your problem one of Silicon or one of Carbon?

In my 20-year career in working with software, the single biggest issue has also been an elephant in the room: the abysmal success rate of large software systems. Plenty of books have been written on the subject with lots of great, if unheeded, advice. Scope creep, ambiguous or missing requirements, poor communications, bad risk management; lots of reasons have been documented and just as many if not more solutions have been suggested. Brooks gave us "The Mythical Man-Month", Yourdon writes of "The Death March", Tom DeMarco has "Peopleware". All of them attempting to answer the question, "Why is it so hard to write software?"

If you take the all the books on writing software and put their suggestions in one big list, the first question would be, "If we have all the answers, why don't we use them?" That might be nice for getting it off your chest but isn't has helpful in pointing us in the right direction. At its core, this question does have a useful kernel, it assumes that 'we' are the problem. If only we used this process, if only we implemented that practice, if only we hired the right people, if only we kept our teams motivated. As an aside, most of the reasons for failure are management reasons, not technical ones. It should be no surprise that many find the case study on the sinking of the Vasa in 1628 to be very instructive. The blame is as much, if not more so, on the King's unrealistic demands as on the builder's efforts to failure to meet the royal requirements. In our endless efforts to build bigger, more complex and complicated software solutions in the face of almost certain failure, should question our sanity; one definition of insanity being, "doing the same thing and expecting different results." Biting off more than we can chew, choking, and then doing it all again time after time only shows how our hubris can get the better of us, allowing us to continue on our Sisyphean task.

This brings me to an interesting observation. There is a long running debate over whether computer programming is/should be considered a form of artistic expression or of engineering prowess. Computers have such a wide scope that many sub-fields have crept up, hardware and software being the most obvious. Hardware is typically the realm of the engineer, designing circuit boards or embedded sensors. This changes as the hardware gets larger in size or scope. CPUs have microcode, missiles have on-board navigation systems with programmable targets.

Most universities that offer degrees related to programming computers name the department Computer Science. Now, being considered a 'science' may be just a way of expressing the notion that computers are deterministic and worthy of comparison with mathematics or engineering, having likely grown as an offshoot of the field of electrical engineering. Whether it be data mining, security, networking, or pure algorithms, most of the areas seem to ignore the question of why software is so hard. They assume the problem is in the silicon computer chips. Curiously, these fields don't even attempt to address this most difficult of questions, assuming improvements of our silicon constructs or their usage will somehow mitigate our human foibles. If only we had better tools, or faster computers, etc. then our carbon-based brains could somehow ascend above the project management miasma and become a beacon to those who struggle and toil without end. Considering the gargantuan number of man-hours and humongous amounts of money wasted, such an accomplishment would be of heroic proportions, mythic even; a modern-day retelling of Prometheus or Sisyphus.

The truth is that Computer Science may bring improvements in areas where the problem is in the silicon, it is mute when asked to provide answers to problems of a carbon nature. Software Engineering attempts to develop solutions that include how humans work together. Software Engineering does not assume that software development is as cut-and-dried as putting the right circuits on a board to achieve the desire product, or breaking down a complex problem into constituent algorithms to make the problem more manageable. There is no magic bullet, no single development process, no single answer. To understand why humans have a hard time developing software, we must admit that humans are part of the equation. Doing is brings with it a cold truth about humans - they are non-deterministic by nature. Alistair Cockburn describes it as people being non-linear, first order components in software development. To answer the question of what is software development, we might have to take detours through Psychology, and may be surprised to end up working with Philosophy even.

When the Student is Ready, the Teacher Appears

2010-09-02T09:21:00.002-05:00

I remember hearing from a 'higher-up' once at a previous company that we shouldn't do Use Cases. "The customer doesn't like them", something I never heard directly from a customer. Doing a design review with use cases, a domain model, and a set of fleshed-out sequence diagrams was once described as 'over designed'. It was a mystery to me at the time why they would be so ambivalent about better ways to develop software.

Now, in my graduate class at UTA on Software Design Patterns, the assignments are ironically familiar:

project requirements
use cases (3 levels)
domain model
sequence diagrams, design class diagram
pair programming
test driven development

Not even ten years ago, these practices were still gaining acceptance in the industry. A common complaint up until now has been that colleges and universities don't teach these techniques, which only gave the critics more ammunition. Now that has started to change, hopefully the debate about these practices will be about how to best implement them instead of how to dismiss them.

It turns out that the solution to the mystery had nothing to do with the merits of any software development methodology but rather had everything to do with the rejection of in-house software development. This company had a long history of developing their own software and over the years had ended up with one of the largest technology departments I've ever worked in, close to 1,000 people. When the leadership changed, the attitude shifted from "Not built here" to "Don't build it if you can buy it". They didn't value the capability to develop software, so trying to improve the process had become moot. Preferring to buy vs. build is a perfectly reasonable approach. Some company's realize they don't want to be a development shop and make no bones about it. The harm comes when there is a disconnect between management and the teams doing the work. Psychologically, this would fall under cognitive dissonance; having a development shop mindset at the same time the departments actions are turning away from development.

It wasn't a surprise then, when it was announced that they were 'partnering' with a large three-letter company to rewrite their web-site, which happens to be the company's main source of revenue. The team that developed the site was held in high esteem by the department, full of some of the smartest guys and pushing the envelope, all while supporting a truly business-critical app 24/7 with close to zero unscheduled downtime. The site happened to be written in C++, putting the bar high for development skills for recruiting. One of the clues to the mystery was at a monthly meeting of technical leads prior to the partnering decision. Using C++ was being de-emphasized, the reason given was the difficulty in finding qualify developers.

The interesting question becomes, if you devalue software development skills will you lose the ability to monitor your partner and be a fully engaged partner? Inevitably, you'll have to take some consultants word on a technical matter and you'll no longer be a partner but rather a simple client. Keeping your organizations software development skills sharp benefits you even if you don't write the code.

Student, Teach Thyself - A Theoretical Computer Science QnA site

2010-08-09T11:58:00.001-05:00

I've been a fan of Stackoverflow for a while. Jeff Atwood and Joel Spolsky have helped the Internet turn a corner when it comes to helping people find answers to their questions. Google may do an incredible job indexing pages but it would be to no avail if the pages themselves were all junk.

I don't have a ton of time to be on the new site-proposal system called area51.stachexchange.com. During one of my CS grad school classes, I asked a question regarding my schoolwork on the main Stackoverflow site and was underwhelmed by the responses. Since the main site is more for working programmers who typically expect a code snippet as an answer, I figured the proposal for a site on theoretical computer science deserved a shout out.

Now if they would only restart their podcast.

The Battle of Computer Science Training

2010-07-27T10:11:00.113-05:00

There is a constant battle between the corporate and academic worlds. Universities want to teach more theory so students are prepared for a wide range of situations. Companies want graduates with technology-specific training so they can contribute immediately. Who is right? Both and neither.

Equipped with skills

When taking a course in algorithms, you might need to demonstrate some type degree of learning through programming a lab assignment. Its a decent bet that languages the student may not be the best choice to highlight the various algorithms being taught. The student then finds himself learning a new language for the sake of submitting a lab assignment. This is usually done with zero training or exposure and very little support outside of asking classmates or asking for help from the instructor who is eternally busy with other things.

Version control is another topic that shows the rift between the two worlds. Most IT shops require the use of some version control system. Most developers learn enough to get the job done but never really understand it enough to make it an equal partner in the development process. The administrative function is usually lacking. big IT shops can afford to hire VCS admins also have so many repositories that the admin's time is spread too thin unless you happen to be working on the 800-lb gorilla project. Smaller shops muddle through and usually someone takes the admin role upon themselves because they are tired of the situation and are the first to hit their frustration threshold. Not unlike the newlywed dilemma, "If the full sink of dishes bothers you, you wash them."

Since starting grad school, I figured I'd apply my own coding lessons and use version control for my assignments. What an eye-opening experience! Some tools with which I'm familiar already have enough version history. Eclipse has a nice feature where it automatically saves the change history of a file every time you save it. This works for solo Java projects but doesn't for other languages or group projects.

There are two main obstacles to using version control in academia; having to be your own VCS admin/trainer, and the lack of consistency from the university. You have to have the discipline to keep your VCS configured and working but also extend it to whatever environment that might be required. It does you little good if you have your personal repository setup on your own PC when the professor requires assignments to be done on the campus Unix box. The same holds true when a lab is required to be done in a language that your IDE doesn't support. Eclipse has 'C' support through the CDT but if you aren't going to be using it next semester, the time it takes to get it configured and working becomes that much time away from getting the assignment done or studying for the next test. When working on a group project, it also means having to teach your teammates how to use the tool and then hope that they actually use it.

This leads to the second obstacle. Without a departmental policy or professorial dictate, version control usage by students will be as rare has hen's teeth. The university or at least CS department would need to support VCS tools for campus computing environments, at least for administrivia. The professors and instructors would need to make VCS part of the lab requirements.

Don't hold your breath.

There is no impetus by the faculty to standardize on a VCS tool. The sheer number of languages and environments make it unlikely for any consensus to appear. This is odd because CS classes constantly have issues with turning in code and checking for plagiarism/cheating. Looking through the check-in history would be a simple way to verify a student's progress. Code diff tools would make it almost effortless to check for "borrowed" code. There are enough cross-platform tools that it is not a technology problem, it's a human problem.

The fundamental problem is the two worlds are trying to 'train' for different things. In school, technology is a means to an end. Students are being trained to understand how to approach and solve problems. They are being prepared for unknown situations so they can recognize when an approach is guaranteed to fail or maybe has been solved before and a stock algorithm exists. Employers, on the other hand, want people who are trained in the tools they are using to solve their problems. New hires aren't the ones who get to analyze a business problem not to mention prioritizing which problem to address. Managers still like to think of coding like a factory floor, the workers don't get to design the manufacturing process or choose the tools.

Trained to repeat a process

For those of you that like analogies, here is how I'd put this conundrum. College is training blacksmiths that don't make the same thing repeatedly. They can be given a strange piece of hardware and figure out how to fix it. They can be asked to create something new based on ambiguous requests. Business wants someone who is trained to make swords. The enemy is already defined by the leaders, the strategies already determined by the generals.

The problem isn't training sword-smiths, it's wanting nothing but sword-smiths and then discovering you need a telescope, or horseshoes. Technology is still changing at an unheard of pace. Training in the current technology is a recipe for obsolesce. The window of current technology is getting smaller and smaller. COBOL was king-of-the-mountain for 20 years, C/C++ for maybe 15, Java is already on the downhill slope with C# and JavaScipt taking the spotlight and other languages in the wings (Ruby, Python, Scala) or working backstage (Perl, PHP, SQL).

Pick your tools well and train for anything. You'll have your fate in your own hands and not be surprised by the winds of change.

To Buy or Not to Buy Technical Books, That is the Question

2010-05-14T08:33:00.003-05:00

So the rookie coder is working at his new job, fresh out of college, when the team malcontent is overheard a couple of cubes over, "This place is so cheap! They should reimburse me for this technical book. It's a heckuva lot cheaper then a training class." However true that is, it hides a simple choice: keep the book and be out the money, or give up the book and pocket the cash - assuming the boss will foot the bill.

Recently a question on Stackoverflow reminded me of that rookie coder. Being fresh out of college gives you an amazing combination of overconfidence and cluelessness. Without any basis, it's easy to take the office attitude and run with it. You shouldn't have to buy your tools, your company should.

I disagree.

Why? Because they are tools. It's not even a cost thing. A professional should be responsible for knowing which tools they need to be most productive. Tour an airline maintenance facility and check out the workshop. A large area, maybe 1000 sq. ft., with nothing but large toolboxes; each with a name and a lock. The mechanics have to buy all their own tools, think wrench sets and so forth - not large things like engine dollies.

At first it seems a bit silly and redundant but when you start to think about it, it's easy to see that not everyone is going to want to spend the extra money on the high-end tools, or they might want extras of the same set for whatever reason. Also, consider the hassles with sharing, e.g. "Where are the 1/2 inch box-end wrenches?" By not making the tools shared, the 'Tragedy of the commons' is avoided.

Another example is chefs and their knives. You don't touch another chef's knives. The restaurant provides the big things like ovens and stoves, the chefs bring their own knives.

Back in the world of computers, I've brought in extra memory before. I told my manager so he'd know that some of my personal property was in the PC in case it needed to be worked on by the internal techs. I also put a note on the box itself. I would have had a bigger issue if they discouraged bringing your own hardware. Most shops like to provide the box itself because it minimizes their support issues but they shouldn't have any problems with you bringing accessories like monitors, keyboards, mice, etc.

To summarize, change how you think about tools and strive to use whatever makes you better at your job, even if you have to pay for them out of your own pocket. It's the 'Professional" thing to do.

Martin Fowler, Alistair Cockburn, and Optimism

2010-04-21T13:13:00.001-05:00

The effort to make Software Engineering actually live up to the 'Engineering' part has been greatly helped by Martin Fowler's book on refactoring. Martin recently posted about his decision to not participate in an effort by others in the industry, not lacking in optimism, to define a common process for software development.

Martin Fowler declined to get involved in a similar effort and for the same reason. The key quote is,

“Why this is so was primarily crystallized for me by Alistair Cockburn who explained that since people are the central element in software development, and people are inherently non-linear and unpredictable - such an effort is fundamentally doomed.”

While my previous post about Alistair's paper covers much of the same ground I felt it was worthy of an update. There will always be souls who crave consistency just as there will always be salesmen willing to offer the illusion of it. Facing our own nature is one of life's hardest lessons. Some of those who ignore the human factor are optimists seeking to improve their chances of consistency, which has driven many towards the latest and greatest geegaw, a new language, a new GUI, a new design tool. Others are simply deluding themselves in the hopes that if only people would sit up and fly right (read this to mean 'do it their way') all our problems would be solved.

Studying the process of computer programming has been around for decades. Even so, we still have trouble internalizing human foibles - people aren't predictable and follow "standards" about as well as a random-walk. So the most successful way to get people to be consistent is to lead by example, blaze a trail ahead and show them the way. In this manner Extreme Programming was its own worst enemy and its own best cheerleader. By showing how a set of practices work in situ, it created a much-needed focus. The "Waterfall" model was the favorite whipping-boy, essentially dead, and the iterative approach was in its infancy. So XP served as a lightning-rod for discussions on the craft of software development. It provided an example of practices that could be emulated and provided an easy target for the doubters, disbelievers, and denialists; no doubt the negativity coming from the critics was channeling their innate understanding of the darkside of human behavior.

While Extreme Programming hasn't become the standard development model, that doesn't mean it failed. When the history of Software Development is written, XP will be given credit for re-introducing the most important factor; not tools nor process, but people.

Context as a commodity

2010-04-12T12:21:00.000-05:00

How much context do you need to do your job? Can we quantify context in any way?

Jeff Atwood has mentioned that he doesn’t shut his machine down every night. He wants his environment to be as he left it at the end of the previous coding session. Getting all the apps, tools or other windows opened to the right place takes not only CPU time but mental energy, aka context.

Some jobs require no context, for example a bank teller can walk in to work on Tuesday and not need to remember any of the transactions that took place on Monday. Contrast that with a novelist in the middle of writing a new book, let’s say writing the next chapter not editing anything. They need to remember all the characters, their personalities, the existing story, and in what ways the plot threads are to interact in the new material.

Another good contrast is a professional athlete. A pro baseball player doesn’t need to remember what happened in yesterday’s game in order to pitch a strike, hit a home-run, or execute a double-play. The limited context that does appear has to do with the optimizing performance. A pitcher does need to know the preferences and history of the batter who is at the plate, just like the batter needs to know if the pitcher has a wicked slider.

Companies want to be “green” these days and have instituted policies to shutdown computers overnight. This may be pennywise and pound foolish. Here’s why. A consultant gets paid $50/hr and it takes 30 minutes to get their workspace ready after an overnight shutdown lasting from 5pm to 8am the next day, 15 hours. A conservative energy estimate would be if electricity costs $0.25 a kilowatt hour multiplied by 15 hours for a cost of $3.75. So $25 is spent trying to save $3.75. That’s motivation to use the hibernate feature at a minimum. It also is a very simple way to quantify the cost of context.

Can we describe jobs by how much context is required? If so, is there a relationship between the amount of context needed and the average salary? Those are interesting questions, maybe I’ll cover those in a later post.

Programmings Best Kept Secret

2010-01-21T09:30:00.004-06:00

Software development is often compared to engineering or construction which has a hidden and debatable assumption that the source code is the product. One camp has declared that source code is the design . Other camps declare that programming as a branch of applied mathematics; see Dijkstra's "uncomfortable truth" quotes. Extreme Programming as a movement may have failed, it at least served as the flashpoint for a new examination of the nature of programming. Instead of arguing about engineering we can brag about how Agile we are or that we are Software Craftsmen.

Programmers are not engineers, nor are they artists. Even when they are compared to craftsmen, the comparison is always with woodworkers or blacksmiths. Programmers are nothing of the sort, programmers are writers. Danny Thorpe discusses how you might tell what sort of literature your app may be.

Danny is on the right track but when it comes to software, there is an elephant in the room that needs to be discussed in the open. Developers do not produce software, they write code. This means that sometimes they need the logic skills of an engineer but just as often they need the communication skills of a poet. Reading and writing, thinking about how to achieve a desired affect through the written word, those are the qualities of a writer. It just so happens that the one of the chosen audiences is a single-minded overly-literal pendantic polyglot idiot-savant, otherwise known as a computer. The other audience is other humans that may or may not have to actually follow the instructions, they do have to understand them well enough to tell if the instructions are correct.

It would be useful to compare programming to other fields and see how the comparison holds up. It is not a coincidence that the games industry has adopted the filmmaking model for the production of games. Good choices would be publishing a book, producing a movie, or creating a recipe.

Programming as publishing

Programmers are the authors. The compiler is the printing press. Publishers are the companies that agree to fund the process in hopes of turning a profit. Self publishing would be like creating training manuals for internal use; no profit is involved but the medium is appropriate for other reasons. The book still needs to be edited, proofread, typeset, marketed, etc. I've covered this ground before but there is still much to contemplate.

Programming as filmmaking

Filmmaking is in interesting comparison because it deals with translating the written word into another medium. Programmers are the writers of the scripts and screenplays. The producer is the stakeholder who desires the software. The director is the project lead. Instead of actors and crew we have the compiler which takes our instructions and executes whatever script of commands we provide; call it a screenplay or call it an application, the actions and actors my be different but it's still a set of commands given to actors to be performed.

Programming as cooking

Programmers create a recipe for a given dish. The recipe is not the final product, the result of following the recipe is. You don't buy a cookbook for the fabulous chocolate cake recipe, you buy it because you want to eat the cake. In this case the compiler would be the tools available to the chef; pots and pans, utensils, and the stove/oven. Having the right tools really helps but even the best oven won't correct a recipe that calls for 2 cups of vinegar for the sugar cookies. Eventually software may survive the patent wars and finally be considered on par with a recipe; instructions to a cook in order to produce the desired product.

Summary

Software is actually a mix of disciplines, all trying to contribute towards a common goal. The game industry may have it right, evidenced by them being to busy producing games and not endlessing debating whether their process is broken or seeking the latest silver bullet. They use directors, producers, art and sound specialists, etc. The compiler is their camera, used to capture the intent of the contributors; the CPU the silver screen for which to project the final product for consumption by the audience.

Of Rockstars and Bricklayers

2009-12-29T08:37:00.003-06:00

Today we're going to start with a thought experiment. You are one of the better coders in your company. All the proof I need is that you're reading this. Ok, that may be a little self-serving but it is true that the better coders I know are always reading something about coding. It might be a new language, or a new technique, a programming question and answer site, or even a site that likes to rag on bad code.

Based on how many Code Monkeys you know, it's easy to figure yourself in the top half of programmers, a 'good' coder if you will. An assumption that is easy to make is that the higher your productivity the higher your pay. Is this true? If you base it on simple experience, the answer is no. It's too easy to get a job right out of college and get mediocre raises and then find yourself interveriewing candidates for openings in your own group where you have to offer 'market rates' to attract anybody. If you've been there for 5+ years, given a company desperate to hire warm bodies, it doesn't take much for the new guy to be making 15-25% more than you even if he is a two-finger typist and can't spell SQL if you spotted him the 'S' and the 'Q'.

Now, I'm happy where I am but it didn't take much to trigger my bad habit of thinking up 'what-ifs'. The Stackoverflow podcast has been a good one for this sort of pondering. Jeff and Joel have enough real-world experience to bring a ton of wisdom to the discussion on programming as it's own topic and in episode #42 discuss underperforming programmers. I'd been reading about how star actors can command huge salaries based on their past work and the expected effect on the box-office if their name appeared on the marquee. That major-league sports can have individuals who make 10-times what the lowest paid athlete makes on a single team, it doesn't take much to ask why programming doesn't see the same effect.

I submitted this as a question to the Stackoverflow podcast which they spent a good 10 minutes on in episode 77. It didn't fit so well as a question on Stackoverflow itself, I suspect the culprit is the lack of code snippets. Interestingly enough I discovered the podcast when they were at episode 65 and decided to start listening back at #1 and was up to around #42 when I decided to contribute my question. So anyway, when I saw the summary for #77, I didn't have the audio handy so jumped over to the transcript wiki for #77 and read the transcript. It was a pleasant surprise at how engaged they were and that the question wasn't dismissed out-of-hand. When I downloaded the audio, to hear their discussion it only reinforced my reaction.

The range of talent can be mighty wide even if there is some disagreement on the empirical numbers. Joel Spolsky talks about the difficulty in finding great programmers, and uses similiar metaphores in a comparison with opera singers. Jeff has a similar take on Coding Horror. Finding great talent is hard because there are only so many people who can do what you need done.

There have been some good discussions that has surfaced. John Cook has a good argument based on lack of meaningful metrics. It's a question that has been around for awhile. Frank Wiles makes the claim that hiring average (or below) programmers actually costs your company money (followup post). Obviously when a company says they only hire the to 10%, they are deluding themselves or at least paying lip-service to wanting good programmers. Joel does a great job of shattering that particular illusion.

What about sales? In some fields top salesmen can bring home seven-figure incomes so we know that the wide range of incomes aren't limited to Hollywood and the big leagues. James Kwak picks up on John's post and supports Joel and Jeff contention that paying the top performers much more than the average, outside of easily measured areas like sales and movies, has a corrosive effect on the group and the best coders end up leaving for greener pastures while the dregs hang on for dear life or are eventually pushed out. I can't disagree. It seems that if you are underpaid you have to ask yourself Ann Lander's question, "Am I better off leaving or staying put?" Just realize that it's no use complaining to the company, the status quo works for them.

Life's Too Short To Work For Clowns.

2009-12-21T14:53:00.007-06:00

One of the cliche's of our culture is the little boy who rebels against his parents and dreams of running away to join the circus. His mind is filled with the glamor of the Big Top, the exotic animals, the independence of wandering the country from one town to the next. He hasn't really thought about what it means to work in a circus. The Ringmaster did many menial jobs before arriving in the spotlight. The animals have to be cared for; double the time it takes to feed them when you consider both ends of the animal. Traveling from town to town may sound romantic until you realize it means you spend more time traveling than in one place, lots of lonely nights on the road regardless of the weather or climate because 'the show must go on" after all. The life of a circus is not just popcorn, parades, and pretty girls.

I ran across a quote in Christopher Duncan's, "The Career Programmer: Guerilla Tactics for an Imperfect World" that made me both laugh and sigh.

You need to consider one additional thing when trying to convince management the current approach to developing software needs improvement. What do you do if non of this sways them and they insist on continuing with chaotic development approaches that consistently create disasters and stress? Update your resume. Life's too short to work for clowns.

So the question becomes, do you work for clowns? Does the development process at your company make a circus look like a well-oiled machine? You might work for a clown.

I plan on putting together a complement to The Joel Test where instead of a list of things a company does before you consider working for them, The Clown Test would be a list of things that indicate you might work for clowns.

This might describe your boss,

if he thinks an estimate is a promise,
if he thinks developers are interchangeable with no effect on the schedule,
if he thinks its ok for the lead developer to publicly berate a junior developer,
if he thinks requirements documents are a waste of time,
if he thinks the only place for designers is at a fashion show,
if he thinks he just needs one good programmer who can code anything,
if he thinks that people whine too much about teamwork,
if he believes in Silver Bullets,

he might be a Tech Clown.

My apologies to Jeff Foxworthy.

When you run across a good example of a Tech Clown, send 'em my way and I'll add it to the list for every one's enjoyment.

The Myth of the Aging Developer

2009-12-11T09:13:00.000-06:00

I question that I've heard throughout my career is, "What happens to older software developers?" The danger with this question is when the discussion turns to the implied version which is "The lack of older developers means that it is a young persons game." Nothing could be further from the truth..

My wife and I are both developers. We have both been working as developers for 15+ years. One thing we noticed was that there were a lot of people working as developers who really shouldn't be, they didn't have the aptitude, interest or sometimes even the background. It would be easy to dismiss the problem as a lack of proper education but that doesn't explain the developers who lack a CS degree but who can code circles around veteran devs. Those folks usually decide that programming is not for them and move on to different job. Some go into management, some become analysts, while others change careers completely.

What's left? The ones who are passionate about development and are interested in doing it right. They love learning new stuff and don't gripe about how things used to be in "The good ole' days." They might be grumpy but they usually know their stuff.

The main problem would be in finding management who doesn't have a pre-conceived notion about what makes a good developer. If they feel that only the younger ones have the stamina to keep up with the pace, I'd wonder whether those same employers were the same ones who believe that the 'Deathmarch' style of project management is standard operating procedure. They'd rather grind through fresh-meat developers (Code-Monkeys anyone?) than listen to the experienced ones and change their development process much less change their management style.

Lastly, I've noticed that doing nothing but coding holds less business value than being able to communicate with customers, users, management, and vendors. One reason you start coding less is because finding answers to ambiguous requirements takes more experience and provides more payback to the business.

My advice, to those who worry about their coding future, would be to find a place where they need experienced developers to help with the overall task of creating or maintaining software, that also gives you the opportunity to code regularly. That way you can have the time to keep up with industry changes while taking advantage of your understanding of software development.

What I learned about Cryptography

2009-12-08T07:18:00.000-06:00

In preparing a presentation on cryptography for a graduate class, I was surprised by what I learned.

Cryptography has been used from ancient times through the Industrial Age to the modern day. It's been used for military, diplomatic, criminal, and commercial purposes.

In the simplest and oldest form, Greeks used strips of paper wrapped around sticks of a certain size called a scytale - rhymes with Italy.

Julius Caesar is one of the first known users of the simple letter substitution method. The modern day cousin is ROT13.

If he had anything confidential to say, he wrote it in cipher, that is, by so changing the order of the letters of the alphabet, that not a word could be made out. If anyone wishes to decipher these, and get at their meaning, he must substitute the fourth letter of the alphabet, namely D, for A, and so with the others.
—Suetonius, Life of Julius Caesar

Hiding messages (steganography) is as old as the ancients. An example would be a tattoo on a shaved head. The hair would grow back hiding the tattoo. A modern example is hiding text in the pixels of a digital image.

Charles Babbage is a name taught in every into to computer science class as a pioneer in computing machines. I never knew that he helped the English during the Crimean War by breaking cryptological codes.

Mary "Queen of Scots" was caught when one of her secret codes were broken.

The Voynich Manuscript may have been an elaborate hoax to defraud a king, who had a interest in mysteries like coded messages, out of a large amount of gold.

A mob boss used encrypted messages to give out orders to his lieutenants and was arrested when the police cracked his amateur code.

Cryptography is thousands of years old and is used all around us everyday.

Programmers - Poets or Mathematicians?

2009-12-03T08:52:00.007-06:00

Programming software is much like writing a story. Any given story can be told in a near-infinite number of ways and those who practice the craft will spend just as much time, if not more, arguing over the style of a work, as if their subjective opinion can suddenly become objective fact. Usually this leads to a contest of "If you don't agree with me, I'll just keep talking until you see reason" or otherwise known as talking the subject to death.

I thought I'd shine some light on an example of this issue in programming. There is a never-ending debate in programming circles about the 'proper' way to write code that creates some true/false decision. Stackoverflow has a good example of this.

My personal pet peeve (petty but my teeth grind everytime I see it) is verbosely setting booleans, e.g.
bool isValid;
if (percentage >= 0 && percentage <= 100)
  isValid = true;
else
  isValid = false;
whats wrong with
bool isValid = percentage >= 0 && percentage <= 100;
It's soooooo much more succinct and easier on the eye

Even though the single line version isn't the most complex, different people read through code in different ways. A single expression can be easily parsed by people who have a math background or are experienced coders. That's not the point. The expanded version reads more like a sentence than a formula. It's easy to forget that there are plenty of coders without a background in math. It really comes back to the preferred style of the reader. Do they prefer reading expressions and formulae or something more like human speech.

Homer's The Illiad and The Oddessy are ancient works dedicated to expressions of speech. They were originally poems that were passed down in an oral tradition and meant to be sung instead of read.

Euclid's Elements is a work dedicated to expressions of logic. It covers objective proofs and unambiguous facts and is the beginning of the mathematical tradition, if you will, of rigorous scientific thinking.

The problem we face in software is code is a form of human expression, with all the imprecision that entails, attempting to communicate with the utterly logical CPU; programmers will be forever trying to translate "An Ode to the Machine" into the Pythagorean Theorem.

If Gulliver Were a Designer

2009-12-01T08:27:00.007-06:00

The term 'Design' gets thrown around a lot in software developer circles. There are constant flamewars over favorite techniques and battles over what good design really entails. When those arguments start getting too ad hominem figure out whether the speaker is a Code Monkey before giving them much credence. Since I've already braved the waters, see "Have Design Will Travel" or "The Age of the Design Review", instead of safely avoiding the issue, I'm going to bravely (or foolishly) join the fray.

Realize that there is more than one type of 'Design'. Design in the large turns into architecture and is something that really only comes with experience. Designing at the lower levels, say at the individual class/object level is easier to cover. Today I'm writing about design in the small vs the large; Lilliput vs. Brobdingnag.

The issues with designing a class is the same regardless of platform or language. The key is whether an object should be autonomous or whether it is better for any given behavior to be spread out among objects with limited scope and distributed responsibility.

For each class the answer might be different. We end up with a spectrum along which we can place classes based upon the Density of Responsibility.

             (Level of responsibility for behavior)
        Autonomy - - - - - - - - - - -  - - Dependence
     High
 C      -   [god object]                [spaghetti]
 l      -
 a      -
 s      -                                
 s      -           
        -                  
 s      -
 i      -
 z      -
 e      -  [template]                   [framework]
      low

Let's say you favor letting the class perform all the behaviors itself, or as many as you can. Starting on the left side of this graph, when you make your class more autonomous, the size of the class will grow unless you continuously refactor it to make it more generic. This leads to a template. If no refactoring is done, the tendency is for the class to become more "god-like" because if there is some behavior it needs, it has a method for that - it can do anything and everything. The number of fields and methods grow and soon become both unmanageable and unavoidable. Since the class already does so much, coders would rather add to the monstrosity than try to piece it apart and cut the Gordian knot.

The right side of the graph has classes that depend on other classes to a large degree. If the dependency level is high but the individual class is small, that is a sign of a framework; each class doesn't do much and requires lots of dependent classes to accomplish some function. On the other hand, a highly-dependent class that also has a large amount of code is a sign that the class is full of spaghetti.

The key to this question is to determine where you feel more comfortable on the graph. In any event, individual classes will end up spread out on the graph unless some organizational principle is applied, which is how you can achieve the results of Template or Framework.

Having just written that, I would say that there is a correlation between class size and degree of organization. Robert C. Martin (or "Uncle Bob") covers similar ground with package dependencies in his very thorough paper on Design Principles and Design Patterns[pdf]. JDepend is an implementation of the ideas behind the graph on page 26 and complements static analysis tools such as Checkstyle and PMD.

Extreme Programming is People

2009-11-02T05:19:00.001-06:00

There has been an enormous about written in the blogosphere on ways to improve how we write software. Much of the virtual ink spilled on the Internet about Extreme Programming, or Agile, or SCRUM, gathers at opposite ends of the spectrum. Some focuses on how those practices make things better, usually in a overly enthusiastic way which invites the curt dismissal as nothing more than hype and hyperbole. Others constantly remind everyone that they've tried it and it didn't work which invites observations about anti-social programmers.

Both camps need to call a truce. They'll get much farther - and be happier - if they stop throwing grenades at each other and start talking to each other.

We can get both sides to agree that the previous ways of software development is problematic and certainly non-optimal, at least if the goal is to develop acceptable software in a reasonably quick, consistent, and effective manner.

Once we've agreed on common goals, we can discuss typical obstacles and plan ways to avoid or mitigate them. One of the biggest problems, and least discussed, is how people affect the processes in which they participate. There is a good article by Alistair Cockburn about this phenomenon. His paper is titled "Characterizing people as non-linear, first-order components in software development".

The Story of Joe and Al

So let's examine more of the laws and how they affect an effort to introduce Extreme Programming. People who use the 48 Laws as a handbook for how to live their lives are usually label as evil, justifiably so. I don't want this discussion to degenerate into name calling or dismissed because of guilt by association so let's create two personas as our antagonist and protagonist. We'll call the willing and manipulative one Joseph (think Stalin) and the unwitting and hapless one Al (think Flowers for Algernon).

Law #1 "Never Outshine The Master". Humans are sensitive about losing face in front of others. Joseph uses this to his advantage to keep from attracting the wrong kind of attention. Al is proud of his achievements and doesn't understand why others don't share in his celebration.

Law #2 "Never put too Much Trust in Friends, Learn how to use Enemies" - Joseph keeps reminding himself to watch out for number one. He doesn't want to put his fate into the hands of others which means fewer allies but also fewer fair-weather friends. Al, on the other hand, cannot comprehend the idea that people have hidden motives even while having repeated instances of being taken advantage of.

Law #3 "Conceal your Intentions" is "Do unto others before they have a chance to do it to you". Keep the knife hidden from your intended victim. Joseph believes that life is a zero-sum game. Since he values himself over all others, he won't lose any sleep taking what he wants so he'll have more. He's interested in power and control, if he takes it from you it means you'll be powerless to respond after his power grab is fate de'compli. Al doesn't have the mental capacity to think one thing while saying or acting another. George Orwell's "Doublespeak" would be a foreign concept.

Law #7 "Get others to do the Work for you, but Always Take the Credit". Some poor soul on StackOverflow called this a "Seagull Manager - a scavenger who takes all the credit for your hard earned work and hangs around you like a bad smell." This will be a common behavior for Joseph who lacks scruples and thinks those who let him get away with it deserve to have their credit stolen. Al believes what society preaches in public, the value of hard work, work hard and keep your nose clean and you'll get what's coming to you, etc. He has the pride of the workman where good work will be recognized and rewarded. Combine that with no ability to detect deceit in others and you end up with someone who becomes the pawn of others. Naively believing the promises of their antagonists.

You can't approach a social situation as if everyone has the same motivations. Workplaces have plenty of Josephs blending in waiting for any situation to turn to their own personal advantage, and Algernons trying in vain to be accepted. If we don't recognize that, we'll be like farmers watering their crops with beer, "It tastes good to me so it must also be good for growing corn."

Chapter 5 - The Programming Team

2009-10-28T06:34:00.002-05:00

This continues my review of "The Psychology of Computer Programming" by Gerald M. Weinberg.

So far we've had individual programmers either working solo or in an environment where other programmers work but on separate tasks. As long as a single person can maintain the conception of the program, conflict resolution involves the thinking of a single individual, regardless how much advice is available or heeded. When a problem is too large for one person to hold in their mind at once, the scale of the problem goes up due to the increasing communication needs but is now one of a different kind. In this second case,

"conflicting technical demands are translated into potential interpersonal conflicts, and a social mechanism must be formed to resolve them"

This is part of the territory. Two people with solutions that are mutually exclusive. Programmers are proud of their intelligence and are more than willing to argue their side.

How A Team Forms

"As a rough rule, three programmers organized into a team can do only twice the work of a single programmer of the same ability"

I'm no longer as convinced of this as I once was. It really depends on how well defined the separation is between the modules.

Establishing And Accepting Goals

"Social psychologists have verified in other contexts that failure of one or more members to share the group goals affects the group performance -- not only through that member's share but through a reduced performance on the part of the others, for they invariably perceive the division within the group or the indifference of one of the members."

A non-programming example of this is when football teams get a new coach; the TV pundits like to use the phrase "They've bought into the new system" meaning that the players have taken the coaches goals for their own. A common programming example you might see in a development shop is where the current toolset and technology stack used is from Microsoft and there is that one guy who keeps muttering under his breath that "Microsoft sucks". His lack of cooperation will be felt by the rest of the team. The result may be that they avoid him instead of sharing the new trick they learned to solve some daily annoyance. No one wants to pair-program with him. The price being paid, at a minimum, is the reduced productivity of the frustrated Microsoft-hater.

The Team In Crisis

"The replacement of a leader--or any team member--is probably the most frequent and typical crisis in the life of a team. The effect of adding or removing a member depends quite sensitively on the group structure, and many a manager has been unpleasantly surprised at the change in performance resulting from the removal or addition of an 'insignificant' group member. "

If we need proof that programming is a social activity, we need look no further. Unless you are the sole programmer working at a company or on a project, there will always be a social aspect to what you do. I'll go even farther and say that unless you are the only person who will use the application, there will inevitably be people involved, from the person who asked you to write the app to the person who will be using the app.

"A member who is competent but who does not get along with the others can be an even more serious problem for the democratic group than an out-and-out incompetent. In an authoritarian group, such a member would not have much contact with others on a working basis anyway, so as long as he gets along adequately with the leader, he presents no particular problem. Indeed, some programmers prefer to work under a strong, centralized leader in order that they do not have to socialize with their fellow workers. But in a democratic team, an antisocial member cuts lines of communications and is a constant impediment to consensus in team meetings."

Sometimes we are our own worst enemy. If we want to stay our of our own way, we need to remember in software development, people are the biggest cause of failure.

Summary

Questions

For Managers:

- Are your hiring practices such that you get more uniformity on teams that you would like? When making up teams, do you try to see that a good "mix" of people is on each one, or do you strive in the opposite direction?

- Do you ever do things to try to inflate the appearance of your technical competence in front of the people who work for you? Describe some of these incidents, and also some incidents in which it was discovered that your technical competence was in at least one respect inferieor to one of the people who work for you. What were the consequences of that discovery, and do they justify attempting to cover up?

- In setting your own working goals, what part is set by what is passed down from above, and what part is set by what comes up from below? Are you satisfied with this arrangement, or would you like to alter it in some ways?

For Programmers:

- What part of programming work do you do best? Are you permitted to contribute that best part of your work to your team, and is it generally recognized that it is the best part of your work?

- Has your manager ever done anything to make you doubt his honesty? If so, describe the incident, what ultimately happened to your doubt, and how your work was affected.

Peanut Butter Programming

2009-10-01T09:09:00.006-05:00

When I first started messing with dynamic web pages it was using Perl and CGI. It was fun at first getting a page to respond based on some algorithm. It was also easy to overlook the growing mound of HTML strings embedded in your program. It soon became apparent that this approach doesn't scale very well. Keeping track of which tag needed to be closed and when was too much like keeping track of dynamic memory allocations in C.

Java came along with Java Servlet Pages (JSPs) and it seemed like a huge improvement. By inverting the relationship between code and HTML, you didn't have to have your HTML all strung out in little bits in the program. When you wanted to include some program logic in your page, you had two choices: write an extension so your logic could be expressed like an HTML tag, or you could embed it as a code-snippet directly in with the HTML. While there were plenty of examples of how to create your own tags, the overwhelming amount of JSP code I've seen took the other approach. To the extreme. When some behavior was needed in multiple places, sometimes it refactored into a common file and then referred to from multiple pages. Other times (which translates to whenever possible), it was copied and pasted around with the justification of "I just need it to work". The result was a file with little bits of program strung out through your HTML. Sometimes mixing two things can prove to be a good thing. remember the the TV commercial for peanut butter cups? "You have your peanut butter in my chocolate! No, you have your chocolate in my peanut butter!" Unlike that TV commercial, mixing HTML and code is a recipe for heartburn. Peanut butter may be tasty but will stick to everything it touches and you'll end up with a greasy mess to clean up if you're not careful. That's why I call this "Peanut Butter Programming".

Why do developers keep using approaches like 'cut-n-paste' even though they usually know better? When the choice is between doing the expedient thing and the right, people typically want to do the right thing but only to a point. If 'cut-n-paste' is simpler or easier than doing the right thing, the playing field becomes so tilted that some other motivation needs to be strong enough to deter choosing the short-term solution. Code standards are one example of that kind of motivation. They express a value judgment that the increasing maintenance costs of supporting bad code is not worth the time saved by taking short-cuts. If the programmer knows he'll have to face his peers and explain why he took a short-cut, his choice becomes "I can do it quick-and-dirty (QaD) in 5 minutes instead of the right way which would take 15 minutes. But if I do it QaD, I'll have to spend 30 minutes arguing at the code-review. It's not worth the argument so I'll just do it the right way and save myself the trouble." This sort of balancing act shows up naturally during pair-programming where having the second coder around makes it harder to justify writing messy code on the first pass.

Recently, I went to a class on Microsoft's ASP.NET technology. Setting aside any feelings for the company, the way they approached the problem of mixing code and HTML was to make it easier to do the right thing. They introduced the concept of a separate 'Code Behind' file which contains the C# or VB code for a given page. By making it easy to manipulate objects on the page and respond to events while keeping the code separate, they've made the right way also be the easiest way.

Even experienced developers will justify taking a short-cut, however short-sighted, if doing the right thing is hard enough to do relative to the short-cut. Different people have different tolerance level for short-cuts. Understanding why people do the things they do isn't easy but gives us the perspective needed to address the problem.

Software as Literature

2009-09-21T09:56:00.008-05:00

Alternative title: Sonnets to CPUs

Have you ever noticed that code follows some basic patterns? JUnit uses Setup(), Test(), Teardown().

I’ve playfully mixed writing and coding below.


//[Play about A Monarch's Destiny, in three acts]
Public KingdomFuture tellTheTaleOfMyRule(Kingdom landMyFatherLeftMe)
{
   //[1st Act:  Build Drama]
   Vassal mySubjects = SoliloquyAboutMyKingdom.getMyPeasants();
   Boolean mySubjectsLoyalty = testOfLoyalty(mySubject);

   Vassal theCaptainOfTheGuard = SoliloquyAboutMyKingdom.getMyTrustedRightHandMan();
   Boolean myTrustedServantsLoyalty = testOfLoyalty(theCaptainOfTheGuard);

   KingdomFuture historysTaleOfMyRule = KingdomFuture.UNDECIDED;

   //[2nd Act: Drama plays out]
   if ( mySubjectsLoyalty == TRUE 
        && myTrustedServantsLoyalty == TRUE)
   {
       HeapPraiseUpon(theCaptainOfTheGuard);
       historysTaleOfMyRule = ExpandKingdomByForce(mySubjects);
   }
   if ( mySubjectsLoyalty == FALSE 
        && myTrustedServantsLoyalty == FALSE)
   {
       ArrestTraitor(theCaptainOfTheGuard);
       historysTaleOfMyRule = ImposeDispicableLaws(mySubjects);
   }

   //[3rd Act: resolve drama]
   if ( deceptionHiddenSuccessfully(theCaptainOfTheGuard) )
   {
       return oldRulerOverthrown;
   }
   else
   { 
      return historysTaleOfMyRule;
   }
}

Discontinuous Integration

2009-09-14T09:23:00.002-05:00

Refactoring is becoming a commonplace term thrown around these days. The expectation is building that refactoring is something that all good developers do and if you don't, you must be in one of "those" types of companies. You know the ones, they don't use a version control system or have a repeatable build, i.e. Continuous Integration, much less a well-defined deployment procedure.

If you are lucky enough to be building the first version of a product, you can refactor to your hearts content. There is no existing code to worry about breaking. If you are changing an existing system refactoring isn't always simple or straightforward. Even if you have the most automated CI and 100% code coverage of unit tests you'll still face the practical issues of deployment and business dependencies.

Let's say you operated the worlds most modern airline reservation system. It's written in the latest language and has all the buzzwords, highly available, highly reliable, highly scalable, etc. It has tons of automated tests so you are 100% confident that nothing will break. Now, since you have business partners like travel agencies and customers looking to book a flight or retrieve flight information, they have a stake in your deployment. The customers may tolerage a "Down for maintenencem, come back later." message but your partners will take a dim view of your downtime. They'll insist you tell them in advance so they can prepare any necessary changes in their systems or alert their support staff in case there are issues after the deployment.

Another example would be a large back-office accounting system. Whatever level of confidence you have in your code. you still have to deal with the myriad inputs and outputs of the system and disrupting the processing schedule for a code migration. No amount of automated testing/build/migration will ever solve these types of complexities.

The point is that considerations like these limit the opportunities to deploy code changes and increase the cost of change even when that change is guaranteed to work.

Refactoring is good but there are practical considerations why it's not done to the degree it could be in some shops. Keep that in mind next time you want to pass judgement on the perceived lack of refactoring somewhere.

In Search of Wisdom

2009-08-31T09:45:00.006-05:00

The other day I was reading StackOverflow, a forum for developers to ask and answer technical questions, and ran across a post from a discouraged student. The responses had one thing in common: programmers know that they must keep learning. Tomorrow's hot new technology is just as soon to be yesterday's lost and forgotten relic. The best programmers are not the experts in a given niche but the ones who are comfortable with learning as a way of life. They know that they do not know and accept it. This makes them resistant to dictated dogma; a good example being Joel Spolsky's "Emperor's New Clothes" treatment of C-style strings. They are then freed from the ups and downs of the technological tide and are not battered by the constant waves of the newest, latest fad, nor do they feel deserted when some favored tech is washed away into the sea or wrecked upon the shoals.

Socrates is said to have walked the streets of Athens looking for a wise man. Wisdom is usually associated with knowledge but since it is impossible to know everything, true Wisdom is the awareness that you don't know. You can know a little about some things or a lot about a few. Those who believe they do know are blinded by their own ego and are thus willing to dismiss all doubts and set a course powered by a self-fulfilling sense of certainty. This describes how many a tribe of men met their tragic fate. Technology is not immune from the siren's song of certainty hence the never-ending search for the always-elusive "Silver Bullet".

When it comes to Wisdom, what sets apart the Codewright from the Code Monkey? A healthy dose of self-doubt. Doubt enough to realize that risks should not be dismissed solely because they are improbable. Doubt enough to allow the most skillful to help do the work, the most far-sighted to help guard the course, the most cynical to help figure the schedule. Doubt enough to keep unjustified self-confidence from enabling acts of hubris. This is just as true for teams as well. The best teams are the ones who understand both their strengths and their weaknesses. A group of individual superstars who can't learn to play together isn't always guaranteed success while a bunch of unknown amateurs who work has a cohesive unit can perform miracles.

Software development as a field naturally attracts those who are willing to accept the reality that there will always be more to learn. It is a good sign, one that indicates our chosen field will never stagnate. There will always be something to discover, a constantly flowing river of new challenges to overcome and puzzles to solve. The search for Wisdom is not for a desired destination, but of a fulfilling journey.

Tower of Babbling

2009-07-15T16:52:00.003-05:00

When I started coding at my first job almost 20 years ago, it was at an OS/2 shop using C and Presentation Manager. I had several different languages in college so I wasn't too phased by the 1-2 week C training course we got. What I wasn't prepared for was memory management, C style string handling, message-driven applications, just to name a few.

I began my dislike of C early with the insanity called string handling. I was pleased when I found out that Joel agrees with me. Although there are plenty of people who will defend how 'intuitive' the string functions are, I knew there were much simpler ways to manipulate strings as long as you didn't need one larger than 255 characters, in which case using a char* buffer would be fine. Writing apps in something other than C was dismissed offhand with statements like, "C is much closer to the metal", "You don't have the run-time bloat of other languages", or "You have so much more control with C" which almost sounds like a masculinity test or something. Think lumberjacks and "my chainsaw is bigger so only the strongest can use it" type argument. Nowadays, this type of behavior would remind me of The Code Monkey.

Even 2 decades ago, it was possible to call code written in one language to call code written in another; in school we called assembly graphics routines from within Ada code for one class assignment. Notable exceptions are running compiled code from within scripts or executing a system command from within compiled code; but rarely do we write a library in C++ to use in our Java app. When Java first appeared and it was still slow, I never understood the resistance to writing the main app in Java and moving the bottleneck code into some C/C++ dll and called using JNI. It's like we use the "language X sucks at a, b, and c so we can't use it" as a shield that keeps us from having to design multi-language applications.

So, after all these years, what keeps us from writing multi-language apps?

An Apprentice of His Very Own

2009-07-06T09:19:00.001-05:00

In my career as a developer, I reached a point where the amount of work that needed to be done was much greater than I had time to do myself. Budget realities aside, there was a lot of coding tasks like writing more unit tests or keeping various diagrams up-to-date that would be tedious but easy to accomplish as long as a basic understanding of development was present. Wouldn't it be nice to be able to afford to hire your own apprentice? Someone maybe still in college who had some exposure to coding but was still wet behind the ears enough to keep quiet and learn by osmosis.

At the time, I had visions of a helper who could take rough UML diagrams, like from scratch paper or from a white-board session, and update the project documentation. As each change came through, they would create new versions and archive old ones like a file clerk from decades ago.

First off, a fresh-from-college developer might consider that type of work as beneath him or her. I'd argue that learning the processes is just as valuable as learning how to code. Did they learn about version control, shared builds or managing a large code-base in school? Of course not, but experience with those types of processes are what most junior developers lack.

Secondly, if your job is consistently repeatable enough to explain to an apprentice, why are you there in the first place? The apprentice is much cheaper and easier to replace. More than likely, you job is one of those ones which is hard to explain because it relies on your experience and training to understand how to approach the problem and the possible/practical solutions.

Usually I stay away from comparing software with construction but in this respect, iron workers have a similar problem: how to bring on new blood without getting anyone hurt or jeopardizing the project. Even the most junior iron worker needs to understand how to do his small part and stay safe. The head iron worker needs to trust that his men won't need explicit instructions for every task and that when a crisis erupts, they'll understand what the priorities are without having to stop and have a meeting.

So, what you end up needing is someone who wants to learn, has the patience to listen, the intelligence to follow directions, the determination to keep doing hard work even when they don't know the importance of their contribution, and the wisdom to understand that not all knowledge comes from a book. The next step is to think about how to carve up your work so someone can help you with it. That will have to wait for another day.