Seeking the exact answers

“No battle plan survives contact with the enemy.”
— Helmuth von Moltke

Sooner or later you will have the business asking questions like “How much is it going to cost?”, “What will I get?” and “When can I have it?”.  The honest answer to this which often provokes a reaction of surprise are: “How much do you want to spend?”, “Tell us what you want” and “As long as it takes” respectively.

Technology projects are complex and high risk, and in particular there is high uncertainty especially before any actual development taking place.  The Cone of Uncertainty [McConnell] describes the uncertainty in estimates of software development project cost, effort, and duration, throughout the lifecycle of a project.   The degree of uncertainty is very large at the beginning of the project, which is often when decisions are made about project pricing and contracts which results in unrealistic project plans and expectations.

Agile or traditional, estimates on duration and costs are inherently very vague at the beginning of a project.  For some complex projects, this uncertainty can last much later in the development process.

We are so accustomed to fixed price, time & cost projects, but in reality they are anything but.  None of this information is new.  However, project sponsors consistently latch on to the initial estimates project managers propose for the schedule and cost of a project at the start when the least is known and when there is the highest variability.  And because project sponsors don’t understand project complexity and other factors influencing cost and schedule when requirements change and emerge, they may see the project as a failure if costs increased, even if the changes improved the value delivered to the business.  Unfortunately, business executives’ lack of understanding about project management influences their perception of IT project success and failure [CIO].

There is inherent uncertainty, even in well-run projects.  This is especially true in projects that involve complex software, large systems, large development effort, new application domains, new users.  Furthermore, unlike manufacturing or building a house software development is a creative process and largely knowledge work which does not fit a cookie cutter model.  We need to recognize that the Cone of Uncertainty implies substantial risks for traditional fixed-price and fixed-scope projects.  The Cone of Uncertainty indicates we cannot predict the future accurately and that we need to restrain from traditional thinking.  If you can’t predict the future, don’t plan it in detail.

A barrier to agile is the lack of the comforting predictive detailed plan whereby we plan out the uncertainty in a project, plot the path, determine the end date and the cost then produce the nice reports along the way.  Interestingly enough whenever we question “are those plans ever right?”  the response is often negative.  Yet we place so much faith in them and they absolutely have to be there because it would be too scary to proceed without one.

Agile practitioners will do an order of less magnitude less effort to come up with a plan and provide answers to the questions “How much is it going to cost?”, “What will I get?” and “When can I have it?” that are just as good and just as inaccurate as traditional approaches.

We strive to ask for exact answers when exact answers don’t exist (estimates are just an approximate judgment or calculation – it’s a guess, but we seem to forget that).    Reality will come to light soon enough.  Striving for exact answers typically results in both time and money being wasted with little improvement in accuracy of the estimates to show for it.

An agile project will come up with a baseline view of how much the project will cost and when it is likely to be finished.  The only difference to traditional thinking is that agile is very explicit right from the start that this is not accurate and the team will strive to understand more as soon as we start working it.  We want to get to the actual development effort where we regularly deliver high quality, working software that maximizes the business’ ROI.

There is an amazing feeling of relief when agile teams can become comfortable with uncertainty and a freedom to actually discover what is needed at the appropriate time.  Instead of relentlessly following the pre-defined plan agile teams will adapt to the inevitably changing conditions.

Being on time, on budget and to specification means nothing when we have delivered the wrong product with low quality that has no business value.

So rather seeking the exact answers, should we be asking if we are delivering business value?

For Fun – PMI Agile Response

An article called the ‘Sweet Spot’ appeared in the August 2010 edition of the PMI.ORG PM Network Journal.

In response to this article, a video was produced which is quite amusing.  Watch the video.

The Mythical Man-Month Is Not A Myth

Brooks’ law is a principle in software development which says that “adding manpower to a late software project makes it later”.  It was coined by Fred Brooks in his well renowed 1975 book The Mythical Man-Month.

There was a recent project where unrealistic timelines were imposed on a team and the project was running late largely due to the Cone of Uncertainty – commitments were made early when least was known.  The team estimated the project would complete in 12 months. However, the project needed to be complete much earlier than that.  As a result, the project showed all the symptoms of a “death march” from day one – desperate attempts to right the course of the project by asking team members to work especially grueling hours, weekends (and public holidays), often causing burnout and by attempting to throw (enough) bodies at the problem with indifferent results.   Interestingly, the project was still delivered in 12 months despite all the overtime and adding more resources.

As an observer from the outside, it amazes me how many people overlook the negative productivity impact of actually adding more resources.  On face value, we seem to think if we double the team size it will simply mean we would double the workload of the team.

In reality, it takes some time for the people added to a project to become productive. Brooks calls this the “ramp up” time.  Software projects are complex engineering endeavors, and new workers on the project must first become educated about the work that has preceded them.  This education requires diverting resources already working on the project, temporarily diminishing their productivity while the new workers are not yet contributing meaningfully.  Each new worker also needs to integrate with a team composed of multiple engineers who must educate the new worker in their area of expertise in the code base, day by day.

In addition to reducing the contribution of experienced workers (because of the need to train), new workers may even have negative contributions – for example, if they introduce bugs that move the project further from completion.

Communication overheads increase as the number of people increases.  The number of different communication channels increases along with the square of the number of people; doubling the number of people results in four times as many different conversations. Everyone working on the same task needs to keep in sync, so as more people are added they spend more time trying to find out what everyone else is doing.

To reduce cost on the project, a significant number of resources were obtained from an offshore location which made things more complicated and accentuated the above issues. Due to reduced communication bandwidth, collaboration was even lower with the offshore team.

A senior external  consultant was bought in to help the team with some new technology.  He commended  the (original) team about the high quality code which enabled him to do his work effectively and to extend and maintain the code with low cost.  However, all the hard work the team originally put in that resulted in high quality and clean code were being lost with the addition of the new resources.  The need for new resources to be coding from day one resulted in poor quality output.  Not only did this result in increased defects, but also increased integration issues.

In my blog Don’t rely on overtime to salvage a plan last month I wrote about avoiding daily or frequent overtime as it affects efficiency, productivity, morale and quality.  Asking the team for mandatory overtime is a symptom of a serious problem on a project and does not address the root cause.  Don’t get me wrong, you can work some short and specific overtime and this can often help get the project out of a tight spot. However, if you come into work frequently and say –

“To meet our goals, we’ll have to work late again”, then you already have a problem that can’t be solved by working more hours”. (Beck, 2004)

So with constantly increasing scope but with a unrealistic fixed schedule, quality was being impacted.  Corners that were cut due to unrealistic pressures imposed on the team and excessive overtime came back to haunt the team later.  They were finding or making more bugs than before resulting in more delays and greater integration issues due to poor quality of work being provided by the offshore team.

Overtime just doesn’t yield the results promised by Gannt Charts which provide an oversimplified view of the software development process by only measuring the quantity of the hours spent on tasks without also taking into consideration the quality of the time (yes there is a difference).

Sometimes the biggest gains in productivity come from stopping to think about what you’re doing, why you’re doing it, and whether it’s a good idea. We need to work smarter, not harder and have the ability to hear unwelcome news and then address the real issues.

When trying to add new team members to address a tactical and fire-fighting need, we need to realize that an incremental person who, when added to a project makes it take more, not less time – nine women cannot make a baby in one month.

Traditional Project Management Role on a Scrum Project

A “good read” article on Agile Journal, An Agile PM Isn’t What You Think Sub-Head: Where Does Traditional Project Management Fit in an Agile Project Using Scrum

There is always considerable discussion about the role of the Traditional Project Manager on an Agile Scrum Project and this article describes it very well.

In the end, Scrum does not prescribe where the individual currently serving as project manager should go in a Scrum project, only with whom the project management responsibilities lie and, as such, organizations should not assume that project managers always belong in a particular Scrum role.

[Edit 16 Nov 2009]

In another article 10 things a PM needs to know about Agile, the authors go as far as stating

There is no PM role in Agile

Whilst there is no single person performing the role of a PM, the elements and responsibilities of the traditional PM are performed by all the people on an Agile team.

I thought the following statement is quite true

“but who is going to chase them up if they don’t get their work done on time!?” In reality, this is more a fear than a real risk.

Ultimately, organisations and management need to trust individuals and teams to do their work – that’s why they hired them in the first place.