Software Musings

Knowing Where You’re Going

Posted on: 07/05/2006

There was an article in last month’s IFR Magazine (the self-proclaimed “Magazine for the Accomplished Pilot”) entitled Skip Flight Planning. Flight planning has changed in recent years from being a tedious process concerning lines on maps, analogue computers (a.k.a. slide rules or E6Bs) and pages of arithmetic to become a tedious process of typing waypoints into a computer and allowing a site such as aeroplanner to do the hard work. The article in IFR took this one stage further: it’s unlikely that you’ll travel the route that you planned since a combination of Air-Traffic Control and the weather, two entities over which a pilot has little control, will reroute the flight anyway, so why do a lot of planning? We have more sophisticated devices in the aircraft now to allow us to replan our route and recalculate fuel burns while we’re en route so, apart from getting a good weather briefing on the ground, planning should be minimal.

This is, of course, reflected in the way that we schedule software projects. We used to spend a great deal of time up front (“on the ground”) putting very complex plans together, believing (or at least pretending that we were believing) that this time requirements wouldn’t change on us—that we would proceed effortlessly along the pre-ordained route with the pre-calculated fuel burn before arriving at our destination on time and on budget.

I am told (although I have never been able to find the source) that it was Einstein who said “Insanity is doing the same thing over and over and expecting a different result”. I have worked with software projects for 25 years and repeatedly witnessed the triumph of hope over experience—this next project will be different. Various forms of Agile Programming, up to and including eXtreme Programming have been proposed to reduce the amount of pre-work, accepting that it will probably change anyway, but can we really say that they have made a major impact on most companies’ thinking? I believe not.

Pilots didn’t relinquish their extensive pre-flight planning until genuine tools emerged that are fitted in the aircraft, that can give almost immediate updates on the implications of rerouting decisions and, most importantly, that are trustworthy. eXtreme Programming and its friends come with a process, a limited set of tools and no indication that what those tools say is trustworthy. Of course, some of the agile techniques (e.g., test-driven coding) have an almost immediate feedback but most cannot be understood by the people who make decisions. An airline executive can step into a cockpit where she understands little, but a moving map with terrain features, flight bars, fuel displays, etc. is immediately understandable. And when she sees that these tools allow her aircraft to recalculate routes dynamically and fly much more directly to their destination, thereby saving fuel, she is over the moon.

What is the equivalent device or tool in the eXtreme Programming world? What one thing convinces the executive with little software knowledge that leaving behind the enormous (and unproductive) up-front planning can lead to quicker and better results? Having given Agile Techniques briefing to executives, I don’t believe that it is there.


3 Responses to "Knowing Where You’re Going"

I don’t know much about planning in software projects, but with planning ahead in general, the important thing about the process is that it forces you to IMAGINE, in some detail, what is going to happen. Any kind of work that stimulates the imagination, however “useless” in practical terms, must surely be a Good Thing …isn’t it? For example, I find that forcing myself (or being forced!) systematically to imagine a responsibility that seems daunting, gets rid of most of the anxiety.

I hope you appreciate my avoidance of the split infinitive in that last sentence. The word “systematically” is the key word, there. I mean, if you sit there imagining your future responsibilities in an UNsystematic way, you are just wallowing in your anxiety, not doing something about it.

I’d be interested to hear the gist of your Agile Techniques briefing to executives, Chris, because I believe the huge potential benefit of extreme (or what is more usually now referred to as agile) programming is being lost. Far from doing away with the lengthy up-front planning process and therefore making a software project seem more risky and less appealing to a potential client, the agile approach offers the possiblility of making the subsequent software far more likely to do the job required.

The crux of any software project, big or small (but it obviously matters so much more on big and very big projects), is getting the requirements right and getting the testing right. This is the area most likely to cause projects to be late and go over budget. The reason for this is that over time the requirements keep changing and the main reason for this is that they were not thought through enough to start with. This only becomes a bad thing if the expectation is that requirements have been agreed and critical software design has started.

A different approach can be adopted. The requirement gathering phase begins not by someone going off and writing a detailed requirements specification but by getting together all those who will be involved with and will use the finished product (or their representatives if the numbers are too great – not everybody has to be at all meetings anyway) and getting them talking to each other. In agile language, they will tell each other stories. At each meeting these stories will be written up and posted on the wall. At first there will be much debate and disagreement and some surprises probably for some parties. After a while a pattern will emerge which will reflect more and more the consensus of what the new system will be like. This consensus will be between all the concerned parties so there are likely to be fewer surprises as time goes on. Also all the nonsense stuff, the annoying stuff and the unnecessary feature list can be jettisoned long before it has any ability to blow the project off-course or scupper it completely. Best of all, there is at least now a sporting chance that what is going to be designed will do the job intended.

To keep the project on track technically as well as strategically, these meetings will include designers (at least some of the time.) The meetings will not deliver code or even a design specification (although there is no reason why demos and mockups should not be developed as part of the story telling process.) They will deliver a credible requirements specification and a test specification which includes a matching test for each requirement. The two documents will be produced by representatives of the client and design participants. Both these documents will be reviewed by the meeting participants and when signed off will be a contractual document within the project. Because of this informal and informally documented approach to begin with, the chances of major requirement changes are reduced and the opportunities for coping with minor changes are that much better.

In other words, use the agile approach as a front end to a project where there are lots of stakeholders (there, that’s a good old corporate word the board will understand) and all sorts of risks of designing the wrong thing. Once this ‘filter’ stage has been gone through, the two most important formal documents, the requirements and test specifications, are in place in the audit trail and the let’s-get-on-with-it design stage can begin.


The points you make David align with mine. In explaining agile programming to executives, most of whom have no software background, I tend to use eXtreme Programming as the example because, if they can’t swallow it whole, there are plenty of places to which you can back off.

The initial promise of eXtreme Programming—to flatten to time/cost of change curve so that change added late does not cost a fortune—seems to be dissipating in the claims being made.

The problem that I find is the argument that runs:

1. this large software project has just overrun on cost and time

2. this was because we didn’t do enough planning and requirements capture at the front

3. next time we’ll do more requirements capture at the front and we’ll succeed.

The more accurate and honest argument would be:

1. this large software project has just overrun on cost and time

2. this is because the project took so long to complete that technology and customer requirements changed during the process

3. therefore next time we’ll be more flexible about accepting that inevitable change and we’ll reduce the elapsed time of each part by breaking the project down

The other promise of agile programming—customer value earlier in the development cycle—is also something to which executives respond.

I have a colleague deeply interested in all of this (and the role that open-source code plays in it). Follow the link to BigBadBob’s blog on my front page.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

May 2006
    Jun »


The author of this blog used to be an employee of Nortel. Even when he worked for Nortel the views expressed in the blog did not represent the views of Nortel. Now that he has left, the chances are even smaller that his views match those of Nortel.
%d bloggers like this: