Book Review – Succeeding with Agile: Software Development Using Scrum

I have been a fan of Mike Cohn and his latest book Succeeding with Agile: Software Development Using Scrum is another great book.  He blends a good mix of theoretical and practical techniques drawn from his past experiences.  This book doesn’t show you how the Scrum Framework works, but it is more of a cookbook of good practices and suggestions that you can help you succeed with Scrum.  So if you are new to Scrum or looking for an introduction to Scrum I suggest you read some other sources first such as the Scrum Guide (by Schwaber and Sutherland) or the Scrum Primer (by Deemer, Benefield, Larman and Vodde) and then coming back to this book for tips and advice as you implement Scrum.

I was able to relate to some of Mike’s observations with my own experience and found myself nodding my head in agreement to some of his sections on ‘objections’ and various stories throughout the book.  As a result, the book felt very personable as I felt like I have been there done that.  He does offer some practical advice when dealing with the various ‘objections’ that you may encounter along the way.    He also points out the challenges in adopting and transitioning to agile.  The following is an important quote from the book:

I’ve personally witnessed several failed agile adoptions that could have been prevented. The first was in a company that had spent more than a million dollars on its transition effort. Executives brought in outside trainers and coaches and hired five people into an “Agile Office” to which new Scrum teams could turn for advice. The company’s failure was the result of thinking that the implications of adopting Scrum would be restricted to only the development organization. The executives who initiated this transition thought that educating and supporting developers would be sufficient. They failed to consider how Scrum would touch the work of salespeople, the marketing group, and even the finance department. Without changes to these areas, organizational gravity pulled the company back where it had started.

A common theme throughout the book is that there is no ‘standard’ way of doing agile and that it must be adapted to the organizational context and project.  He devotes a whole chapter, ‘ADAPTing to Scrum’ [Chapter 2] to this topic.

Although it is mainly centered on Scrum, Mike describes various other techniques and methods to produce a handbook that is thorough and complete for any Agile practitioner.  Scattered throughout the book are real-world case studies drawn from the Mike’s experience helping many software organizations adopt Scrum, and ‘things to try now’ sections based on his advice offer practical, quick ways of coming up to speed fast.  I really liked the chapters on New Roles [Chapter 7] and Changed Roles [Chapter 8] where he describes the 3 key Scrum roles but also identifies how the traditional roles changes to fit into an agile development framework.

I highly recommend reading this book as part of your Agile training, whether you are implementing Scrum for the first time or an experienced practitioner.  It is an easy and good read.

The Mythical Man-Month Is Not A Myth

Brooks’ law is a principle in software development which says that “adding manpower to a late software project makes it later”.  It was coined by Fred Brooks in his well renowed 1975 book The Mythical Man-Month.

There was a recent project where unrealistic timelines were imposed on a team and the project was running late largely due to the Cone of Uncertainty – commitments were made early when least was known.  The team estimated the project would complete in 12 months. However, the project needed to be complete much earlier than that.  As a result, the project showed all the symptoms of a “death march” from day one – desperate attempts to right the course of the project by asking team members to work especially grueling hours, weekends (and public holidays), often causing burnout and by attempting to throw (enough) bodies at the problem with indifferent results.   Interestingly, the project was still delivered in 12 months despite all the overtime and adding more resources.

As an observer from the outside, it amazes me how many people overlook the negative productivity impact of actually adding more resources.  On face value, we seem to think if we double the team size it will simply mean we would double the workload of the team.

In reality, it takes some time for the people added to a project to become productive. Brooks calls this the “ramp up” time.  Software projects are complex engineering endeavors, and new workers on the project must first become educated about the work that has preceded them.  This education requires diverting resources already working on the project, temporarily diminishing their productivity while the new workers are not yet contributing meaningfully.  Each new worker also needs to integrate with a team composed of multiple engineers who must educate the new worker in their area of expertise in the code base, day by day.

In addition to reducing the contribution of experienced workers (because of the need to train), new workers may even have negative contributions – for example, if they introduce bugs that move the project further from completion.

Communication overheads increase as the number of people increases.  The number of different communication channels increases along with the square of the number of people; doubling the number of people results in four times as many different conversations. Everyone working on the same task needs to keep in sync, so as more people are added they spend more time trying to find out what everyone else is doing.

To reduce cost on the project, a significant number of resources were obtained from an offshore location which made things more complicated and accentuated the above issues. Due to reduced communication bandwidth, collaboration was even lower with the offshore team.

A senior external  consultant was bought in to help the team with some new technology.  He commended  the (original) team about the high quality code which enabled him to do his work effectively and to extend and maintain the code with low cost.  However, all the hard work the team originally put in that resulted in high quality and clean code were being lost with the addition of the new resources.  The need for new resources to be coding from day one resulted in poor quality output.  Not only did this result in increased defects, but also increased integration issues.

In my blog Don’t rely on overtime to salvage a plan last month I wrote about avoiding daily or frequent overtime as it affects efficiency, productivity, morale and quality.  Asking the team for mandatory overtime is a symptom of a serious problem on a project and does not address the root cause.  Don’t get me wrong, you can work some short and specific overtime and this can often help get the project out of a tight spot. However, if you come into work frequently and say –

“To meet our goals, we’ll have to work late again”, then you already have a problem that can’t be solved by working more hours”. (Beck, 2004)

So with constantly increasing scope but with a unrealistic fixed schedule, quality was being impacted.  Corners that were cut due to unrealistic pressures imposed on the team and excessive overtime came back to haunt the team later.  They were finding or making more bugs than before resulting in more delays and greater integration issues due to poor quality of work being provided by the offshore team.

Overtime just doesn’t yield the results promised by Gannt Charts which provide an oversimplified view of the software development process by only measuring the quantity of the hours spent on tasks without also taking into consideration the quality of the time (yes there is a difference).

Sometimes the biggest gains in productivity come from stopping to think about what you’re doing, why you’re doing it, and whether it’s a good idea. We need to work smarter, not harder and have the ability to hear unwelcome news and then address the real issues.

When trying to add new team members to address a tactical and fire-fighting need, we need to realize that an incremental person who, when added to a project makes it take more, not less time – nine women cannot make a baby in one month.

As Work Changes, So Must Managers

Here is an article published in The Wall Street Journal earlier this year which is very relevant to today’s rapidly changing environment and the role of managers on Agile Projects.
(Mark Hurd who was my former CEO at HP is mentioned in the article)

What is a manager?

In simplest terms, a manager is someone who organizes a group of people to accomplish a goal. It is a job as old as the human race.

As an academic discipline, management is much younger. Frederick Winslow Taylor is often cited as the founder of management studies. His 1911 book “The Principles of Scientific Management” portrays managers as organizers: they arranged cogs in the industrial machine. Their job was about increasing efficiency and productivity. For Taylor, management “studies” meant standing in a workplace with a stopwatch, measuring workers’ actions, and devising ways to eliminate “all false movements, slow movements and useless movements.”

But in the years since World War II, the nature of work has changed. Peter Drucker was the first to clearly capture the difference.

In 1959, Drucker used the phrase “knowledge worker” – referring to people whose work primarily involves the manipulation of information and knowledge, rather than manual labor. The knowledge worker’s contribution to an enterprise couldn’t be measured with a stopwatch or a punch card. It couldn’t be forced or controlled by any amount of oversight. And it couldn’t be encouraged by simple pay schemes tied to hourly output.

Carlos Ghosn, CEO of Nissan Motor Corp., explains how he got buy-in for major layoffs in Nissan’s turnaround. From an interview 11/17/09.

Drucker broke the manager’s job down into five pieces. A manager, he wrote:

Sets objectives. He or she is responsible for determining what the overall objective of the group is, sets goals for each member of the group, and decides what needs to be done to reach those goals and objectives.

Organizes. He or she divides the work into achievable chunks, and decides who must do what.

Motivates and communicates. The manager creates a team out of the workers, so that they can work together smoothly toward a common goal.

Measures. The manager creates yardsticks and targets and determines whether they are achieved.

Finally, a manager develops people. In Drucker’s world, people aren’t interchangeable cogs; they are individuals who must be trained and developed to achieve the full power of the organization.

To be a good manager in today’s world, you must also be a good leader. It is not enough to give employees directions. Managers must also give them purpose.

What exactly does that mean? In his 1989 book, author Warren Bennis listed differences between managers and leaders. Among them:

  • The manager administers; the leader innovates.
  • The manager focuses on systems and structure; the leader focuses on people.
  • The manager relies on control; the leader inspires trust.
  • The manager has a short-range view; the leader has a long-range perspective.
  • The manager asks how and when; the leader asks what and why.
  • The manager imitates; the leader originates.
  • The manager accepts the status quo; the leader challenges it.
  • The manager does things right; the leader does the right thing.
  • The modern manager’s challenge is that you must do all of the above. It’s an awesome but essential task. Managing without leading is a recipe for failure.

Attempting to lead without actually managing is disastrous as well. Many managers have met their downfall by setting an ambitious vision for their organization, and then assuming someone else would execute it.

“One characteristic of the successful dreamers I think of – Francis Ford Coppola, Steve Jobs – is that they also have remarkably deep understanding of the industry they work in and the people they lead, and they often are willing to get very deep in the weeds,” writes Stanford Professor Robert Sutton. “This ability to go back and forth between the little details and the big picture is also evident in the behavior of some of the leaders I admire most,” he says, mentioning Anne Mulcahy, chairman of Xerox; Bill George, former CEO of Medtronic, and Mark Hurd, the CEO of Hewlett-Packard.

These truths don’t just apply to CEOs. They also apply to the millions of middle managers who make up the core of any large organization.

Middle managers, of course, are not masters of their own fate. Unlike a Mulcahy or a Hurd or a Jobs, they must carry out an agenda that someone else has set for them. They may bear a heavy load of responsibility, but they have limited room for freedom of action.

A decade ago, the Journal’s Jonathan Kaufman captured the challenges of modern middle management by shadowing the manager of an Au Bon Pain bakery caf named Richard Thibeault. The 46-year-old Mr. Thibeault, a former factory worker, had always thought becoming a “manager” would mean he had arrived in the world. He would sit behind a desk, work 9 to 5, and be a pillar of the community.

Instead, he found he had to rise each day at 3 a.m. to bake muffins, prepare soups and fret over his store’s falling sales. Instead of the steady hours he enjoyed in the factory, he often put in as much as 70 hours a week. His job was an odd mix of broad responsibility and limited authority. He trimmed staff in order to meet corporate cost-cutting targets, but was not allowed to cut prices in order to attract needed customers.

“Some days I think maybe I should go back to factory work,” he told Kaufman. “It was easier.”

Yet for all their frustrations, middle managers in today’s well-run organizations often find they are given surprising responsibilities. They may find themselves heading up a team whose tasks involve not just following orders, but solving a knotty problem or developing a new product. They will be asked to innovate, to challenge the status quo.

Middle managers often find themselves heading projects that involve others who don’t directly report to them. Giving orders won’t cut it. Middle managers, even more than CEOs, must exercise influence without clear authority.

They must learn, in other words, to be leaders – the topic of our next column.

Managing Defects on Agile Projects

This recent Dilbert comic strip reminded me of a situation when a client was raising cosmetic defect as severity 2.

It is often not clear what a defect is and at what level to raise it at (despite agreed severity descriptions) as my example above shows.  And some defects are raised without being related to a requirement. Or defects raised because of a feature being “not user friendly”.

Regardless of the reason and how you label the ‘defect’, I treat all defects the same – it is a piece of work required by the client that needs to be prioritized with other work. So the approach is to put all work (defect, enhancement, new feature etc) into the product backlog. As part of the planning process, the Product Owner will prioritize the items on the backlog for the next iteration and/or release. So if a cosmetic defect is more important (has greater business value) than a new feature, then it should be placed higher so it gets completed first.

Keeping all work in the product backlog, makes planning simpler and helps manage the flow of work. It also ensures the client is in control and be the judge of what is important to their business.

Don’t rely on overtime to salvage a plan

I came across a nice analogy today in Mike Cohn’s Succeeding with Agile book which describes what sustainable pace means:

Watch any marathon, and each runner will keep it up for 26.2 miles.  Look more closely, however,  and you’ll notice that the pace is not entirely consistent from mile to mile.  Each works a little harder going up the hill and maybe recovers slightly coming down it.  At the finish line, most accelerate and sprint at a pace that is not sustainable beyond the finish line.

Sustainable pace should mean the same to a Scrum team:  Most of the time the team runs at a nice, even pace, but every now and then team members need to kick it up a gear, such as when nearing a finish line or perhaps attacking a critical, user-reported defect.

Working overtime occasionally does not violate the goal of working at a sustainable pace.   Some overtime is justified.  Do overtime only when we are in critical situation but do not to make it daily or frequent practice as it affects efficiency, productivity, morale and quality.  Consistent and extended overtime is a symptom of a serious problem on the project. It is even more serious when overtime is scheduled into a plan.

Teams cannot be pushed infinitely hard and beyond a certain point, working more hours in a week will move the team backward rather than forward.

Don’t automatically assume overtime equates to actual progress or delivery of value. Often overtime just means that people are present longer, but they aren’t working more. Many studies, in different industries, have shown overtime hours to be far less productive than normal working hours.

When overtime is consistently required, we need to address the cause not just the symptoms. It is a learning opportunity of our capability to schedule and rethink how we reschedule. What is it about our scheduling approach that produced an incorrect plan? Agile is about inspecting and adapting our approach and learning as we go along.

Waterfall Alliance

[Humor] Details of the Waterfall Alliance was released on 1 April 2010, which is driven by the values and principles of the Waterfall Manifesto for Realistic Software Development.

Some great resources can be found on the Waterfall Alliance website.  Further news of the Waterfall 2010 conference has been delayed (yet again).

There is no ‘Best Practice’ in Agile

Joe Townsend has written an article along the lines what I have been saying for a while now – in Agile there is no ‘best practice’, and as Townsend puts it

What works for you, your team, division, corporation, etc. can bring another person, team, etc. to a screeching halt.

In particular, one needs to be very careful when trying to take waterfall best practices and trying to apply it to an agile context. Many of waterfall concepts and best practices are counter-productive and ineffective on agile projects.

Its only semantics, but the term ‘best practice’ often means this is the best way to do things and trumps all other approaches. I prefer the use the words ‘lesson learned’ or ‘guidance’. What one or more teams have done, should be used as a lesson learned that is to be adapted to the environment you are applying it to and used as guidance.  No practice will work for everyone or every team in every context.

Before applying a lesson learned or guidance, you should ask yourself does it make sense and does it align with agile values and principles? Does it help me deliver software more quickly to my customers? Does it help with achieving technical excellence? Remember to discuss with the team before using any lesson learned or guidance.

Large-Scale Agile

Despite that we are still yet to finish the current scope of work for the Agile Initiatives, we are starting to discuss future items in our backlog – distributed Agile and Agile@Scale.

There are a few projects in flight that couple of colleagues are working on that are large Agile programs of work.  Large-Scale Agile by James Shore provides an insightful and thoughtful approach.

On a past project we used a similar technique mentioned in James Shore’s post to Using Bounded Contexts to Minimize Dependencies and Using Kanban to Manage Cross-Team Workflow. Though we didn’t use Kanban specifically, the outcome of the approach was similar – when the Team A is ready to work on something new, they took the next item off of the backlog, work on it until it’s done, and then delivered it to Team B that requested it.  Where this feature is prioritized in Team A’s backlog is dependent on when Team B needed it.

Agile as a way to profitable growth

Not only should Agile or Lean concepts be adopted for projects, but it should also be harnessed at an organizational level to be more competitive.

By focusing more on the bottom line and ignoring the human capital it was one of the reasons What Killed GM:

General Motors managers became very numbers obsessed and bottom-line driven – to the detriment of innovation, creativity and ultimately the product itself.

While GM was obsessing over the bottom-line, it missed the boat on creating automobiles that more people would buy – and love.

And it goes further to say

Lean means more than simply cutting costs or streamlining. Lean, as successfully applied to manufacturing, means doing things “simpler, faster, better, cheaper”.  Notice that the last item on the list is cheaper. If you adopt a systems perspective into every business process. You find where the waste is and you drive it out, focusing on doing things faster and with higher quality, cost will naturally be driven out of the system.

In lean IT, the focus is on collaborative teamwork—represented by all parts of the business—to deliberatively and systematically tackle problems. Right now, IT is forced to fight fires every day. The focus of lean IT is to put forth “a set of principles that says you are going to slow down in order to speed up”.

In a recent article, SOA, Agile and Cloud will pave the way to Lean IT, states that:

in fact, cost-cutting itself, the prime mission of every C-level executive these days, essentially becomes a secondary consideration of the lean approach. As organizations adopt lean methodologies and practices, streamlined costs become a natural byproduct of the process.

In the wreckage of a 24 month ERP/ CMS/ Web project delivery at Lonely Planet (so waterfall the program reviews were known as the ‘Nuremberg Trials’), the seeds of an Agile IT organization, coupled with an ITIL focused operations team were adopted to bring Lonely Planet back from the brink of IT. In this case, Agile was adopted not for strategic purposes but for bare necessity and survival.  Now under BBC ownership, Agile is a well ingrained habit, with new ventures into book product development using Agile and Scrum – a great example how Scrum can be used for non-IT related business process.

There is a lot of benefit for being an Agile and Lean organization.

Bridging IT and business

We sometimes hear that business is not getting value out of IT.  Technology and tools are constantly improving but we often have missed expectations.  The need to align business and IT is getting more important.

Incorrect assumptions of what the client needs often leads to incorrect actions resulting in wasted effort.

Next time you  speak to your customer, clarify what they mean and speak in their language.  Get the customer to provide real life examples and elicit feedback along the way.  That way you will move towards aligning IT with the business and deliver greater value.

%d bloggers like this: