Agilability

Hold Your Horses, Young Guns

2011-10-09T21:21:00.005-04:00

Last week, I had an opportunity to attend a short talk by Jeff McKenna. Jeff was involved in the first Scrum project (you know, Easel Corporation, Smalltalk…). It was very interesting hearing his perspective on Scrum's early days. One thing in particular that I found intriguing was that he said all they were trying to do on that first project was to figure out something that would work. That confirmed a hunch of mine that Scrum's practicalities were realized before people started thinking about how Scrum relates to complexity theory, etc. That isn't the story you might infer from reading the literature.

I was also impressed with the respect he mentioned for Don Reitersten's work, and his recognition of the growing awareness of Lean throughout the Agile community. He made it a point of emphasis that Lean preceded Scrum by decades. I think I chuckled out loud when he said that, because that's a thought that has always crossed my mind whenever I've heard someone touting Lean as something newer and better than Agile.

Of course, Jeff said a lot of other things that resonated with me. But more so than any one thing that he said, it occurred to me that with so many people in the Agile and Lean communities competing to be the next generation of "thought leaders", we have a wealth of knowledge and experience in Jeff and his cohorts that we'd be ill advised to ignore. To hear some tell it, that group is just a bunch of curmudgeons who are no longer in touch. Having had the pleasure of spending a few minutes with Jeff, I have to say that I strongly disagree.

T-shaped Agile

2011-09-29T05:46:00.003-04:00

Quite frequently, I hear folks in the Agile community splitting hairs over their favorite approaches to delivering products and services. There are those who will argue over whether or not one should be allowed to mention "Scrum" and "Lean" in the same paragraph, on the basis that Scrum's focus is too narrow. And then there are those who proclaim kanban to be a higher-evolved technique than Scrum, while others argue, with just as much conviction, that kanban is for teams that can't self-organize very well. I've even heard some people question if XP is, technically-speaking, "Agile" since, by prescribing specific practices, it may be at odds with the spirit of the Agile Manifesto (true story). Yes, it seems that in the midst of all our excitement at actually being able to produce working, valuable products, we've come to have a whole new generation of process bigots.

The blindly zealous promotion of one tool or framework, or methodology, or whatever, over all the others is short-sighted, and it can make our community look foolish at times. From my perspective, the lines between different agile approaches have become very blurred. They all promote letting people work together to produce products that work and are valuable with as little non-value-added stuff as possible, they all include mechanisms for continuous improvement, and they all include practices that can be traced to Lean thinking.

Many of us refer to "T-shaped people" as a way to describe team members who are very deep in some particular area of expertise, but who are also competent in complimentary skills. This concept can also apply, in my opinion, when it comes to our agile approach to providing value to our customers. Thinking in terms of a "T-shaped approach" makes sense to me. I can't remember the last time I saw a team that is successful at Scrum that didn't take advantage of user stories and at least some engineering practices from XP. And the widespread practice of Scrumban is surely an indicator that those two approaches aren't mutually exclusive, although some implementations of that are more heavy on the "Scrum" than the "ban", and others just the opposite.

Does this mean that we should no longer formally teach the rudiments of Scrum or XP, or that we shouldn't seek an understanding of Lean? Not at all, because without that, we'd have no basis for the "deep" part of the T-shape, and just end up with collections of cherry-picked practices of convenience (such collections are notorious for their failure rates). But neither should we stop innovating. Possibly, the two most important words in the Agile Manifesto are "are uncovering".

Anymore, I get the ball rolling with a team by asking, simply, "What is it that, project in and project out, obstructs your ability to rapidly and frequently produce something that is valuable and acceptable to your customer that actually works?" It's usually the same core list of problems, regardless of organization size or the type of business they're in. Based on how things work in their world, though, the appropriate approach for them will be solidly rooted in one proven agile approach, with additional complimentary practices from other proven approaches. Remembering that all of these approaches are more alike than they are different should allow us to concentrate on building good stuff that our customers love, and have a good time doing it. That's a lot more fun than arguing.

Agile Transformation? Really?

2011-09-18T20:21:00.005-04:00

“Agile Transformation”: now there’s a search phrase that can keep you busy on Google for quite a while. But what do we really mean by “transformation”? A quick peek in the dictionary tells us that a transformation involves a change in composition or structure, even a change in character. The word “metamorphosis” is sometimes listed as a synonym.

There is no shortage of folks, myself included, who are willing to help you with your transformation. But if you’re pondering going down that path, it’s a good idea to find out if your organization really wants to change before you get in too deep.

Sometimes, you can pull off a stellar performance in making your case as to why agile would make sense for your organization on many different levels, and the answer (sometimes very explicitly, sometimes not) may still be, “Thanks, but no thanks”. The prospect of making changes to an organization’s composition, structure, or its very character can be intimidating, and to some, frankly, just not worth all the fuss.

Transformation, by definition, entails irreversible change. If we latch onto the “metamorphosis” notion, the implications of transformation become even more stark: Butterflies don’t turn back into caterpillars.

A currently popular approach to making change more embraceable is to advocate an “evolutionary, not revolutionary” approach. Here again, though, the words and techniques we choose can’t and shouldn’t obscure the real objective: permanent change. Slower, maybe, but still permanent. An executive once told me, after I explained this to him, “Well, we don’t really want to become ‘agile’ – just more agile than we are now”. He clearly wasn’t after a transformation. He just wanted to make a few adjustments.

One way we can avoid failed agile transformations is by first correctly assessing whether or not transformation is really the goal. For many organizations, it simply isn’t – at least not right now. They’re content to try to devise viable “butterpillars”, with all the dysfunction you might imagine in such a creature.

This is not to cast a disparaging eye toward such organizations, as many of us have been down that same road to one extent or another. The point here is that a true transformation has no reverse gear, and that for it to succeed, we must be willing and able to make a commitment to changes that are both fundamental and lasting. Sometimes, only after having suffered the consequences of attempting in lieu of committing is an organization finally ready to do so. The organizations that have made that commitment, in my experience, don’t miss the chrysalis at all.

Continuous Integration and Testing: Mere Child's Play

2011-09-18T20:06:00.004-04:00

My youngest son loves to play the game “Mouse Trap”, and last Christmas vacation, I had the opportunity to play many rounds of it with him. The object of the game is to build the mouse trap contraption piece by piece as the players take turns rolling the dice and moving around the game board. One thing he insisted upon was turning the crank and running the machine every time a piece was added — even when the crank wasn’t attached to anything.

Now, I’ve been playing this game for decades, and my vast experience told me that there’s no sense in turning the crank until we had the machine significantly finished. Besides, I had other things I needed to get to, and running the machine every time we added a piece made the game last longer. “We only added one tiny piece,” I’d say. “There’s no reason to test every time. What could go wrong?” At the start of one particular game, though, I decided to do my best to be a patient dad, so I gave in and agreed to do it his way.

As the game played out, with him turning the crank after each piece was added, what I found is that some assumptions we were making when adding a piece weren’t realized. Sometimes, a piece would have been installed backwards. Other times, installing a new piece would knock a preexisting piece out of alignment, and the machine wouldn’t work as it had before. Any time the machine didn’t work, we’d find out where the problem was, make an adjustment, turn the crank again, and make sure it worked before moving on to next roll of the dice.

So, what started out as my indulging his fascination with the machine actually turned out to be him teaching me a thing or two about the value of continuous integration coupled with continuous testing. I was a little embarrassed that I hadn’t thought of it first, and that a 7-year-old had come by it intuitively. My old habits were dying hard. He hadn’t acquired any bad ones yet, and so it made perfect sense to him to test the system for no other reason than that something had changed. This experience solidified the answer I give when someone asks me when a software development team should test: “Any time anything changes”. (In the context of TDD, of course, the answer would be “BEFORE anything changes”.)

This experience also reminded me of how we often resist doing something because it isn’t the way the we’ve always done it, and how leaving the decisions up to the “experts” can result in missed opportunities. I now have serious game when it comes to “Mouse Trap”, and a stronger appreciation of continuous integration and continuous testing — all because I’ve learned to think like a 7-year-old.

Self-direct, Live Long, and Prosper

2011-09-18T19:56:00.003-04:00

I enjoy discussions of the technical theory and mechanics of agile development as much as the next person, but it is the human aspect of what we do that truly intrigues me. I found out about the Whitehall Studies a couple of years ago while watching a documentary on stress. In a nutshell, an initial study of British Civil Service workers was done in 1967 that showed a higher incidence of disease and premature death among those in the lower employment grades. A subsequent study between 1985 and 2004 was conducted to try to figure out why. Turns out it had nothing to do with income, living conditions, diet, or healthcare.

There were a lot of interesting correlations discovered through this study, but one in particular caught my attention. Although conventional wisdom said that executives experienced more stress because of their level of responsibility, it wasn’t the executives who were dying young. The major cause of illness, in fact, was determined not to be high demands, but the combination of high demands and low control.

Just as this lethal mixture described the circumstances under which the subjects of this study were laboring, we don’t have look too far to find organizations (software development or otherwise) whose people are swimming in the same toxic soup. I’ve known for a long time that well-functioning, self-directed teams are highly productive and loads of fun to work on. It might not be a stretch now to say that serving on a self-directed agile team that is empowered to control how it gets its work done may even add years to your life!

What I Learned From My GPS

2009-07-21T11:53:00.009-04:00

I recently took my family on a vacation that required us to drive several hours to a place we had never been. Instead of reaching for my worn road atlas or even using an online trip planner, I decided to rely totally on my GPS. As I prepared to pull out of the driveway, I plugged in my destination and in seconds was informed of how many miles I had in front of me and my estimated time of arrival.

As we rode along, what caught my interest is that each time we exited the interstate for food, etc., the automated voice would announce "Re-routing". Then, after getting back in the car to resume our journey, we'd be given instructions to get us back on track, and the information in the GPS would be updated to reflect the distance remaining and the new (now later) ETA.

I thought, "Wow -- talk about inspecting and adapting". The GPS was constantly comparing my progress to what I had told it I wanted to do, and letting me know how I was doing in terms of distance and time. Any time I did something that deviated from the path that would take me to my destination, it assessed the situation, told me what I needed to do to get back on track, and let me know, matter-of-factly, that based on my rate of progress, my ETA was being pushed further out. It even informed me of traffic obstructions and asked if I wanted to route around them.

On the day we were to start our drive home, I was informed by one of my teenaged “stakeholders” that he HAD to get back that night because he had a day trip planned with his buddies the next day. I had not planned on going all the way home in one night, and I could have played the “you should have told me this while we were planning the trip” card, but I decided to do what I could to get home by 2:00 AM, with the caveat that I would see where we were at 1:00 AM and make the call then. In order for us to make it the required distance in the time allotted, I would have to drink lots of caffeine and drive way too fast. By about 12:30 AM, it became clear that we would not reach our destination on time.

I had a decision to make, then: I could either plod on home, but get there very late and exhausted, or go as far as I could by 2:00 AM. I decided that we had made significant progress and that I only needed to find some place safe for my family to spend the rest of the night. So I reduced the scope of the distance I would cover to that which was feasible given the rate at which I could realistically and safely travel and the time limit I had established. Although one particular stakeholder in the back seat was not at all happy that we would not make it home in time for his trip, it was clear that his requirement was optional and of relatively low priority and that pushing on through the night would create a safety risk that we did not need to take. We found a nice hotel, were in bed by 2:00 AM, and drove the rest of the way home the next day well-rested.

I’ve made similar trips with paper map in one hand and military-grade coffee in the other, and pushed through, taking it one mile marker at a time. What usually happened in those cases is that, with destination and time frame in mind, I overestimated my ability to stay alert, drove myself to exhaustion, and even then still didn’t reach the destination on time. To make matters worse, a couple of those times I’ve found myself out of options, with no place to sleep, and nowhere to get the next cup of mud except some seedy truck stop called “Lulu’s”. The difference this trip was that I was constantly made aware of exactly where I was, how far I had to go, and what my true capabilities were. That information enabled me to make the best decision, all things considered, while there was still time to make such a decision.

I won’t go into a patronizing comparison between paper maps and linear plans or mile markers and milestones, but this trip guided solely by a real-time “information radiator” was full of obvious parallels to my work in agile software delivery. I’m also happy to report that although my son did not make his trip with his buddies, they made up for it by all going to a movie a couple of nights later.

Who's Driving the Bus?

2009-06-15T13:03:00.006-04:00

There are many things I liked about the late Ray Charles. Not only was he a fabulous musician and performer, but he had the capacity to laugh at himself and even capitalize on the fact that he was blind. Ray played a cameo role as a blind bus driver in a movie a few years back, and in his signature scene, he announced, "Next stop, Sunset Boulevard... I guess it's Sunset Boulevard..." Very funny in a comedy, but no so funny if that's the driver for your organization's transition to agility.

The phrase "make sure the right people are on the bus" is a frequently quoted expression in the Agile community that conveys an abundance of wisdom in the context of building and sustaining agile teams. I'd like to suggest that we can expand this metaphor from the team to the organization, and that for an enterprise to successfully transition to agility, it is of supreme importance that the right leader is driving the bus.

Grass-roots or even "middle management" agile initiatives can only go so far until they run into organizational impediments that limit their success and, quite possibly, threaten their long-term survival. This is especially true in larger enterprises with complex organizational structures. In order for such an organization to move beyond the state of perpetual struggle, issues that are almost certain to be politically sensitive will have to be dealt with. This requires leadership that knows where the enterprise is going in terms of its agile practice, understands why it is going there, and is unquestionably positioned in the driver's seat.

Just who this person is will vary from company to company. It could be an individual, but as is often the case with product ownership, the responsibilities of this role might also be performed by a "virtual person" comprised of a handful of tightly-knit individuals whose skills complement each other and who speak with one voice. The important thing is that there is an active and respected apex of leadership that transcends any localized interests.

If "apex of leadership" sounds a little heavy-handed for an agile organization, consider that although agile approaches are bottom-up and team-centered, lightweight frameworks and "rules" have arisen that serve to guide the team. Similarly, high-level organizational leadership can facilitate and expedite during the course of a transition to agility without being dictatorial. Part of that leadership's responsibility does, in fact, include ensuring that "the right people are on the bus". It also includes, among other things, funding adequate training and being committed to responding when organization-level impediments are raised. This still qualifies as servant leadership, in my opinion -- just on a large scale.

Making the Business Case for Agility

2009-05-04T12:27:00.006-04:00

Overview
Gather a bunch of agile practitioners in a room, and the conversation quickly turns to lively theoretical discussions and war stories. Yes, it is truly energizing to be around like-minded people, and we know this stuff really does work. But enthusiasm and anecdotes alone don't carry much weight when trying to convince executive-level management that transitioning to agility would be a good move.

Why Don't They Get It?
We miss opportunities to properly position agility when we try to appeal to senior management with the wrong type of information. We live and breathe the ground-level principles and practices of agility every day, but people in these positions are more likely to be measured according to (and thus, preoccupied with) things that leave them impatient with our explanations of agile concepts, project management frameworks, and engineering practices. When approached with such low-level information, the case for agility often comes across about as convincingly as a child's justification as to why she "needs" a pony. If we hope to be successful in promoting agility to people in the higher levels of an organization (whose support we'll most certainly need), we have to be able to articulate, in straightforward terms, why an agile approach to software delivery makes good business sense.

Review the Basics
Generally speaking, the production of anything involves the conversion of some kind of input into an output that is valuable to someone. An investment is made in raw materials, they are input to a transformation process, and the output is sold to customers who find value in it.

In software development, we don't transform iron ore into steel, or steel into beams. We take ideas and transform them into software that somehow represents the essence of those ideas and allows access to those ideas via features, so that people can manipulate them in ways that benefit them. We have to be careful when comparing software development to manufacturing, but I don't think it is a stretch to treat the ideas as the raw materials, the cost involved with bringing an idea to the state that it can enter the transformation process as the Investment, and the expense incurred in transforming that idea into a functioning feature as Operating Expenses, and the resulting working feature set, IF it provides valuable benefits to the customer, as Sales. This view is consistent with Throughput Accounting theory.

Spend Less, Make More, and Make it More Often
Although a lengthy discussion of Throughput Accounting and Achieved Value (versus "Earned Value") is beyond the scope of this post, the business argument for agility really is founded on these bodies of thought. In my view, the key points as they apply to what we're talking about here boil down to these:

Work in progress, no matter how "complete", only represents an expense, and is of no value to a customer.
Until something is sold, all we've done is spend money.
Shorter cycle times facilitate decreases in Investment and Operating Expenses, and increases in Sales.

We know that reductions in Investment and Operating Expenses, together with an increase in Sales will result in increased Net Profit and Return on Investment. We've also seen that, in software development, expenditures associated with eliciting and preparing ideas for transformation into features constitute our Investment, the costs of transforming those ideas into features are Operating Expenses, and that Sales presumably result when we make valuable benefits available to our customers. In order to maximize the financial performance of the software development organization, then, we can draw some very direct conclusions:

We should minimize expenditures associated with preparing ideas for input into the transformation process, as this will minimize our Investment.
We should minimize the amount of partially-completed work that accumulates in association with a particular deployable set of features, as this will serve to minimize our Operating Expenses.
We should produce what the customers actually value; otherwise, we won't have any Sales.
We should maximize our opportunities to deliver our product, since Sales can only happen then.

The Connection
Getting to this point hasn't required us to invoke any agile terminology. We haven't even used the word "agile". All we've done is cite some basic relationships between Investment, Operating Expenses, and Sales, relate them to a high-level view of the economics of software development, and draw some obvious conclusions. The theory certainly runs deeper than what we've looked at here, but the proposition of making the software development organization more financially successful sets the stage for the discussion of specific agile principles and practices, and why they make sense. For example, it is much easier to propose lightweight requirements documentation, backlog prioritization, and the concept of "Last Responsible Moment" when it is understood that the real objective in doing so is to limit Investment. Similarly, it can be shown that cross-functional teams, limiting work to capacity, and continuous integration introduce system efficiencies that minimize partially-completed work, thus reducing Operating Expenses. And making the case for achieving "Done" in short iterations becomes a less arduous task when it is understood that the underlying rationale is to increase Sales.

And so...
The principles that are woven throughout the various agile frameworks and engineering practices have the effect of limiting Investment and Operating Expenses and producing revenue-generating value frequently, so that the organization can maximize its opportunities to realize Sales against a minimized investment with minimal operating expenses. That is the real message that needs to be conveyed to senior-level management. By establishing the proper business context, we can lay a foundation upon which we can build an agile organization and gain some powerful allies in the process.

Are Some Software Development Challenges Too Complex for Agility?

2009-03-31T14:01:00.012-04:00

Synopsis
When the concepts of agile software development are introduced to an organization, a common response is, “Sure -- that will work with a simple application, but ours is a very complex ‘Enterprise’ system”. Could it be that core agile software development principles are too simplistic to tackle the really tough situations? I recently found myself in a position in which it appeared that this might be so.

The Problem
The problem that confronted me was that I had a legitimate stakeholder requirement, but I was limited in my implementation approach by a highly-interdependent system of systems. To compound matters, the optimal approach that would allow me to mitigate risk to the system while implementing the specified requirement was precluded by a firm stakeholder-imposed constraint. It didn’t take long for the agile-averse to begin chanting, “See how complicated it is? That’s why an agile approach won’t work here. This requires REAL architecture and design”.

So there I was with a justifiable stakeholder requirement, limited by a tightly-coupled architecture, shackled by a constraint that wouldn’t go away, and playing to an antagonistic crowd. It was a fine situation I had gotten myself into, and I was certain that I could get out – I just wasn’t sure how. I tried to think of an analogous set of circumstances that would help provide clarity while distancing myself from the details specific to my situation, but nothing quite captured the essence. After having wrestled with this for the better part of an afternoon and into the night, I decided to call it a day. I flipped on the television for a few minutes of decompression, and happened across a very interesting documentary…

The Story
The story was that of a man who had lived most of his 50 plus years with a medical condition that had severely diminished his ability to breathe, eat, and see, and which had finally progressed to the point that it was life-threatening. He had seen many doctors over his lifetime, and had always opted to live with the condition, but with the realization that his life was in danger, he was ready to seek treatment. The documentary team followed his journey to seek two medical opinions.

The first stop was a highly-qualified surgeon who had years of experience treating similar cases. After examining the man, the surgeon reported that he believed that he could help the man via a major surgical procedure, and that several reconstructive procedures would follow. He went on to inform the man that in order to perform the procedure, a blood transfusion would be required during surgery. For personal reasons, however, the man would not agree to have a blood transfusion. Faced with that fact, the surgeon told the man that he would not be able to help him.

The second opinion was rendered by a team of surgeons at a different institution. Upon evaluation of the man’s condition, they too reported that they would be able to perform the required surgery. However, the procedure they would perform would also require a blood transfusion. Again, the man would not consent. This time, though, instead of simply sending the man on his way, the team got together and came up with a second option. The second proposal was to perform a series of less-invasive procedures. This would provide vast improvement in the man’s ability to breathe, eat, and see without requiring any blood transfusions. The tradeoffs would be that they would not be able to accomplish all that they might have with the major procedure, and they would not be able to do some of the reconstructive surgery. The documentary concluded with the man still having not made a decision between a) going forward with the surgery team’s second proposal, or b) continuing to live with the condition.

The Light Comes On
The team of surgeons had been able to propose a way to meet the man’s most important needs while respecting his fundamental constraint, but balanced that with the reality that the solution would be an incremental delivery of a functionally-sufficient solution, and not a solitary “perfect” solution. It was a fantastically agile approach. As I watched the story unfold, I saw parallels between this and my situation, and it reminded me of some core principles:

Good teams will, and can be trusted to, make good decisions
As this story and standard practice in innovative medical institutions around the world indicate, the best opportunity for providing optimal medical choices derives from team collaboration. This principle works in software development, too, and I believe that the best solutions come from teams, and not gurus or heroes. As with the first surgeon in the story, the limited perspective of a single “expert” can result in too quickly locking onto the trees and missing the forest.

All of the constraints can’t be fixed
There’s certainly nothing new in this statement, but stakeholders will still try to control all the parameters (many project managers, incredibly, will nod yes and then set off to plan it so). Just as the man was left at the end of the story, stakeholders will have to make difficult choices at times. One of our most important responsibilities is to give them the best, most straightforward information possible to assist in those choices.

The desired benefit needs to be kept center-stage, even when it deviates from the “specifications”
Often, the desired benefit can be realized by some means other than the feature as specified by the stakeholders. In this man’s case, the team was able to maintain focus on his most critical needs and present a feasible option for meeting them, despite an immutable constraint. Sometimes there’s a less-expensive and/or more feasible route to the benefit other than the feature as specified. There may be situations in which the oft-cited “80/20” rule comes into play, whereby, for instance, we can offer a solution that would give the stakeholders 80% of what they’re asking for with a reasonable amount of effort, while the other 20% would cost dramatically more.

Problem Solved
Though I had allowed the problem I was facing to be cast as a perplexing dilemma, I realized that the path to its solution was really no different from the one I routinely take with problems I would consider less complex. This wasn’t an issue for one person to figure out, nor was the goal to deliver “to spec”. The resolution to my predicament, then, was to go back to the stakeholders and say, “Let’s put this to the team, realizing that they will propose options that will present the benefit you desire, but that the implementation may not be exactly as you have specified. Understand that the constraint you have imposed may limit our options, but also be reassured that you will be able to see incremental progress early and often, and that we can change direction in between increments as you see fit.”

Final Thoughts
Unquestionably, architecture and design are still important in agility. Dean Leffingwell’s “architectural runway” concept is an excellent approach to this, in my view. Moreover, we certainly need domain and technical expertise if we expect to produce valuable software. And, yes, there are situations in which strict adherence to specifications is required. But a hallmark of agility is that it deals with complexity by simplifying it -- not by fighting complexity with complexity. And now we are seeing industries that were once deemed largely off-limits to agility because of regulatory constraints embracing agility. So, are there software development situations that are too complex for the application of core agile principles? Until someone proves otherwise, I say “no”.