Agile welcomes changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
Short feedback loops in Agile allow the team to get fast feedback and incorporate changes required by the customer and reduce the possibility of the team developing the wrong product for too long.
The Product Backlog is dynamic and evolves over the lifetime of the product and requires constant grooming and refinement. The Product Backlog is continuously updated by the Product Owner to reflect changes in the needs of the customer, new ideas or insights, moves by the competition, technical hurdles that appear, and so forth.
During an iteration, the Agile Team should invest some time to work with the Product Owner to refine the Product Backlog (a.k.a. Product Backlog Grooming). This includes requirements analysis, splitting large items into smaller ones, estimation of new items, and re-estimation of existing items. Priorities can change and requirements can move up and down in the Product Backlog. When new items are discovered, they are inserted into the Product Backlog causing the lower ones to move down the list (and possibly fall outside of the scope of the release) or old requirements removed as appropriate.
Although Agile welcomes changes, when using time-boxed delivery like Scrum, there are no changes allowed during the iteration, and the end date of the iteration does not change. In return for not making changes during the current iteration, the Product Owner can make any changes they want to the Product Backlog before the start of the *next* iteration. The Product Owner can add, remove, reorder, or change items before the start of the next iteration. This enables the Agile team to make and keep commitments, it gives the team focus and stability during the iteration, and it trains the Product Owner to clearly think through what is in the Product Backlog. Due to the short time-boxes, the team should be able to get some reasonable certainty into the planning and requirements and avoid churning during an iteration.
Overtime if you find that the Product Owner keeps needing to change requirements mid-iteration, then perhaps the iteration length is not short enough. Plan iteration durations around how long you can commit to keeping change out of the iteration.
How many times have you heard, “that’s what I said, but not what I meant or wanted”? Too many times I would suspect. Many symptoms of project issues are related to organizational, collaboration, and communication issues, not technology. One of the challenges of any I.T. initiative is getting people to communicate and collaborate together, especially between business and I.T.
Wikipedia defines it as:
Communication is the activity of conveying meaningful information. […] Communication requires that the communicating parties share an area of communicative commonality. The communication process is complete once the receiver has understood the sender.
The last sentence above is quite important and often overlooked. We rely too often on using static channels of communication, such as sending an email or word document to communicate a message. However, the receiver of the message may not have understood what your intention is and we often ask ourselves “why people don’t understand it? I sent them the email with the requirements!”.
The sixth Agile Manifesto principle reminds us:
The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
Getting people together face-to-face provides richness in our communication through, tone of voice, eye contact, visual gestures and the ability to convey tacit knowledge.
Collaboration is working together to achieve a goal. Agile And again, we can look to the Agile Manifesto for some guidance:
Customer collaboration over contract negotiation
Business people and developers must work together daily throughout the project
The best architectures, requirements, and designs emerge from self-organizing teams
During a team discussion, I liked what a fellow colleague said on what collaboration meant:
Collaboration means acknowledging that you don’t have all the answers and that the collective wisdom is better used to solve problems. Collaboration is being vulnerable and being transparent so the conversation brings out things that were otherwise not consciously known.
My communication and collaboration experience
Over the last several weeks I have been coaching and facilitating several inception/discovery workshops that included both business and I.T. One particular workshop was attended by 35 people from all levels of the organization with some participants traveling from interstate. The goal of the 3 day workshop was to discuss and plan how we would tackle a large program of work by identifying the program vision, goals, drivers, measures and the various streams of work that would make up the program.
In the 3 days we accomplished what previously would have taken weeks (possibly months) to complete. The face-to-face workshops provided a rapid, efficient and effective way to for people to communicate and collaborate. An important outcome was obtaining a shared vision amongst all participants. After the 3 days everyone walked away with a shared understanding and vision that they can take into the various projects that would make up the program of work.
What I also observed was the large amount of informal communication and collaboration that occurred outside the workshop hours such as during breaks and lunch that otherwise would not have occurred. Getting everyone together allowed whole team discussions which I believed lead to greater buy-in by all team members.
There was a high level of collaboration between everyone in clarifying business requirements, the problem space and each others understanding. The power of bringing people together to communicate and collaborate to achieve a goal was powerful. Trust was being built between business and I.T.. The business was able to understand the complexities and risk involved in I.T. development. And the level of negotiation between the parties allowed mutual decisions to be made and the quality of the decision making was made possible by having the right people together.
Agile stresses the importance on communication and collaboration. Alistair Cockburn recently remarked how important it is to increase collaboration between people and across the enterprise. Effective communicators realize that the goal is to share information, and that the information sharing is typically a two-way street. Through conversation we can iterate over meaning and seek clarification. The ability to answer questions in real time is important because questions provide insight into how well the information is being understood by the listener. Face to face collaboration also reduces the cycle time of information sharing through continuous flow of tacit knowledge.
Agile is about bringing people together to achieve more than they could individually. It is only if we have consensus through group collaboration that we can provide optimal solutions to address business problems. So next time you want to bridge the business and I.T. gap consider increasing the amount of real-time communication and collaboration between the two parties:
- Get all the players in the one place and talk.
- Engage the right players to make decisions.
- Leverage the knowledge of the team – listen as much as you talk, ask questions when you don’t understand, question if you do not agree. There is no silly question!
- Clarify and iterate over meaning to get a common understanding:
- “So what you’re saying is…”
- “Let’s use an example…”
- “Doesn’t that imply…”
- Find the value that each member adds to the group.
- Think ‘one team’. We stand or fall together. Collaboration expands our potential to succeed.
- Negotiate to reach an understanding, resolve a point of difference, or to produce an agreement upon courses of action.
- Document and capture important information so it is best not forgotten (don’t document just for prosperity sake) – it is the dialogue that is important.
Software development is a creative process and any creative endeavor requiring teams collaborating together to share knowledge, learning, ideas, solve problems and build consensus. Communicating and collaborating will always help you steer your way to a far more successful outcome.
Most people associate daily stand ups or daily Scrums when they think of Agile. And they are often the first practice that is adopted. I have observed some teams adopt the daily stand ups and a little of other practices (often with smells/anti-patterns) and say they are Agile.
The adoption of agile methods and practices is clearly producing enviable results by improving quality, time to market, productivity, increasing transparency and ROI. Agile is simple, but it is not easy. Few things that are truly worth doing are easy to do, and the adoption of Agile is no exception. As a result there is a propensity for Cargo Cult Agile teams looking for a silver bullet.
Cargo Cult refers to a phenomenon during World War II in the South Pacific where the natives observed what the Allied Forces were doing. The natives were not aware of the modern civilization, however, they did observed that certain actions of the military resulted in supplies of food, clothing, equipment and various other cargo being air dropped. When the war ended, the flow of goods and cargo ended too. Not understanding the underlying mechanisms for the delivery of cargo and in an attempt to attract further deliveries, the natives engaged in ritualistic practices such as building crude imitation landing strips, aircraft and radio equipment, and mimicking the behavior that they had observed of the military personnel operating them. However, no matter how well they mimicked the military actions, the cargo did not return.
In Agile we sometimes behave like Cargo Cult by imitating the superficial Agile methods and practices without having any understanding of the underlying essence. We adopt the daily stand up meeting because it is easy to do and many Agile teams use it. However, if the team is synchronized, communicating and collaborating well, the daily stand up may not be required.
What works for you, your team, organization, etc. can bring another person, team, etc. to a screeching halt. No practice will work for everyone or every team in every context. Before adopting any practice or technique, you should ask yourself does it make sense and does it align with the Agile values and principles? Does it help me deliver software more quickly to my customers? Does it help with achieving technical excellence? Will it help my team to be more effective?
You not only need to know the ‘what’ of Agile but also the ‘why’. Just because jumping jacks works for someone else, it may not work for you.
Have you been in a similar situation as the above Dilbert?
Lean and Agile is about doing more with less. Lean and Agile doesn’t focus primarily on cost reduction, but more on strategic alignment around enterprise agility – reducing waste and simplified processes which then could have an impact on cost reductions. Furthermore, for comparable cost as traditional (waterfall) methods Lean and Agile can deliver higher quality product and a greater return on investment (ROI) by working on the most important and highest valuable features first – the business gets greater value for less.
Lean Software Development which has been popularised by Mary Poppendieck is derived from Toyota’s Production System. Basically, lean is centered on adding value with less work. Lean Software Development (Poppendieck) recognizes the following 7 types of waste:
- Partially done work
- Extra processes
- Extra features
- Task switching
Some of these wastes can have a large impact on the amount of value you can deliver. Essentially, by having less of the above wastes we can deliver more. Partially done work means you have unfinished development and resources are tied up on items that provide no business value. Business value is only delivered when features are fully completed. Partially finished features result in hidden costs to projects. When it’s time to release, you have to complete an unpredictable amount of work. This destabilises release planning efforts and prevents teams from meeting commitments, often resulting in more work. Methods like Scrum deliver small increments of fully working software that are usable and tested. Scrum teams agree on a definition of done to ensure they are delivering features that are truly done, not only in terms of functionality but in terms of quality as well.
One big waste often found in waterfall approaches is doing extra steps and tasks that add no value to the actual process of developing working software. Doing extra process steps (such as necessary meetings, extra paper work, manual reports) ties up resources which can be better used on value-added tasks and on activities of producing working software.
Extra features or the overproduction of features is often a result of gold plating. Gold plating as described in Wikipedia:
Gold plating in software engineering refers to continuing to work on a project or task well past the point where the extra effort is worth the value it adds (if any). After having met the requirements, the developer works on further enhancing the product, thinking the customer would be delighted to see additional or more polished features, rather than what was asked for or expected. The customer might be disappointed in the results, and the extra effort by the developer might be futile.
On one of my past projects we had an Architect who wanted the team to build a framework for the system to cater for future needs. This future need may not be realized as things inadvertently change. The framework is a typical example of YAGNI. There was no real requirement (technical or business) for the framework and there was a simpler solution which still satisfied the immediate business needs. The framework added complexity which in return increased maintenance overheads to support it. Every extra piece of code produced will mean more time is needed to test, integrate, compile and maintain and thus increasing the overall cost of ownership. The key to avoiding gold plating is to ensure you only build the bare minimal that is necessary to deliver the greatest business value. If you don’t need it now, don’t build it. Keeping things simple is one of the Agile Manifesto Principles – Simplicity–the art of maximizing the amount of work not done–is essential.
To help avoid extra features, it is important to identify the minimal marketable features (MMFs). Minimal marketable features is the smallest set of functionality that must be realised in order for the customer to perceive value. It is the minimum viable product that has just those features and no more that is fully functional and allows you to ship the product to users so they can start using it. The goal is to build the features that are the highest priority and has the most business value and avoid building unnecessary extra features that are less useful. Agile’s approach of iterative and incremental development and focusing on delivering the most valuable business capabilities takes advantage of Pareto’s Principle which says you will get 80% of the business value from 20% of the effort. This will allow you to deliver more value with less waste.
To complete more work with less time, you should avoid multitasking or task switching. Every time you switch between tasks, you have to completely shift your thinking and gather your thoughts before you can get into the flow of the new task. Multitasking can make you lose focus as it takes you a while to get back and remember where you were. As a result there is a cost each time you change tasks or shift from one project to another. To complete more work, try single tasking.
Waiting is the result of time delays and idle time (time usually waiting for things to happen and where no value is being added). This can be waiting on people, analysis paralysis, delays in approvals, long planning times, delays in testing, and delays in integration and deployments. Lean seeks to optimise the process by reducing the cycle time, which is the time from when an idea is realised to the time it is consumed by the customer.
Agile approaches such as Scrum emphasise the need for cross-functional teams where the team has all the skills it needs to develop and deliver fully working and tested software in each iteration. The reason for this is to reduce the cost of motion and actions of people that do not add value to producing the product (such as traveling from one location to another to find an answer). Furthermore, Scrum emphasises the need for a dedicated Product Owner who can guide the team daily from a business perspective. All team members and customers are often co-located in a team room on Agile projects to reduce the amount of motion needed to find an answer. Frequent face-to-face verbal communication is also important to reduce the motion of moving artifacts and documents. Traditionally, hand-offs of documents between analysts to designers to developers to testers is fraught with waste and mis-communication.
Finally, Defects are a huge waste and takes resources away from producing business value (new business capability) to fixing defects. Instead of fixing defects, Lean principles address this point by focusing on building quality into the process and preventing defects from occurring in the first place. Agile teams deliver working, tested software every one to four weeks. They find themselves making more changes to the code, and staying focused on today’s deadlines. The development team and the code itself must become more Agile, which means it must be capable of faster change, sooner to keep up with business changes. Agile teams have found in practice that Agile code is extensible, low-defect code with the simplest robust design that will work for the features currently implemented. It is well-factored and well-protected by unit tests. Among the Agile methodologies, Extreme Programming (XP) goes into the most depth concerning how developers can keep themselves and their code agile. Such practices enable the team to do more changes and provide them with a kind of Tai Chi flexibility – a new feature, enhancement, or bug can come at the team from any angle, at any time and teams can make the change with confidence that it won’t compromise on the integrity of the system.
Agile development teams regularly reflect by inspecting and adapting their way of working through retrospectives. This enables the team to make incremental improvements regularly and tackle changes in manageable, bite-sized pieces that can be actioned immediately. These retrospectives are important in identifying and eliminating waste in processes which will allow teams to do more valuable work.
In recent times, Kanban systems are being used in software development to limit a team’s work-in-progress to a set capacity and to balance the demand on the team against the throughput of their delivered work. By doing this we can achieve a sustainable pace of development and flushes out issues that impair performance and it challenges a team to focus on resolving those issues in order to maintain a steady flow of work.
The simple act of limiting work-in-progress with kanban encourages higher quality code and greater performance. The combination of improved flow and better quality helps to shorten lead times, improve predictability and due-date performance. [Anderson 01]
Through commitment to Lean and Agile approaches of waste elimination, simplified processes and continuous process improvement teams can deliver in a continuous flow and do more with less. So next time your pointy-haired boss wants you to do more with less, look into how Lean and Agile can help you.
- [Anderson 01] David Anderson, “Kanban: Successful Evolutionary Change for Your Technology Business”, 2010, Blue Hole Press.
If you have been following my blog, you will notice that Dilbert comics seem to jolt me into writing a new entry. In this comic, the pointy-haired boss actually makes a wise comment when he says “Just break into whatever size groups you think make sense”. For Agile teams, what make sense is small teams. I like how Amazon.com refers to their teams as “two-pizza teams” meaning that the team can be fed by 2 large pizzas. If they can eat more than 2 pizzas, then the team is too large.
Scrum guidance states that team sizes should be between 5 and 9 people and this is no co-incidence to the paper The Magical Number Seven, Plus or Minus Two. Smaller teams are more effective because people can’t remember more than 7 things at a time – team members need to know what each other is doing. There are many benefits of smaller teams so if you need to scale up a project, add more teams than making existing teams larger. Remember to keep it to two-pizzas
Brooks’ law is a principle in software development which says that “adding manpower to a late software project makes it later”. It was coined by Fred Brooks in his well renowed 1975 book The Mythical Man-Month.
There was a recent project where unrealistic timelines were imposed on a team and the project was running late largely due to the Cone of Uncertainty – commitments were made early when least was known. The team estimated the project would complete in 12 months. However, the project needed to be complete much earlier than that. As a result, the project showed all the symptoms of a “death march” from day one – desperate attempts to right the course of the project by asking team members to work especially grueling hours, weekends (and public holidays), often causing burnout and by attempting to throw (enough) bodies at the problem with indifferent results. Interestingly, the project was still delivered in 12 months despite all the overtime and adding more resources.
As an observer from the outside, it amazes me how many people overlook the negative productivity impact of actually adding more resources. On face value, we seem to think if we double the team size it will simply mean we would double the workload of the team.
In reality, it takes some time for the people added to a project to become productive. Brooks calls this the “ramp up” time. Software projects are complex engineering endeavors, and new workers on the project must first become educated about the work that has preceded them. This education requires diverting resources already working on the project, temporarily diminishing their productivity while the new workers are not yet contributing meaningfully. Each new worker also needs to integrate with a team composed of multiple engineers who must educate the new worker in their area of expertise in the code base, day by day.
In addition to reducing the contribution of experienced workers (because of the need to train), new workers may even have negative contributions – for example, if they introduce bugs that move the project further from completion.
Communication overheads increase as the number of people increases. The number of different communication channels increases along with the square of the number of people; doubling the number of people results in four times as many different conversations. Everyone working on the same task needs to keep in sync, so as more people are added they spend more time trying to find out what everyone else is doing.
To reduce cost on the project, a significant number of resources were obtained from an offshore location which made things more complicated and accentuated the above issues. Due to reduced communication bandwidth, collaboration was even lower with the offshore team.
A senior external consultant was bought in to help the team with some new technology. He commended the (original) team about the high quality code which enabled him to do his work effectively and to extend and maintain the code with low cost. However, all the hard work the team originally put in that resulted in high quality and clean code were being lost with the addition of the new resources. The need for new resources to be coding from day one resulted in poor quality output. Not only did this result in increased defects, but also increased integration issues.
In my blog Don’t rely on overtime to salvage a plan last month I wrote about avoiding daily or frequent overtime as it affects efficiency, productivity, morale and quality. Asking the team for mandatory overtime is a symptom of a serious problem on a project and does not address the root cause. Don’t get me wrong, you can work some short and specific overtime and this can often help get the project out of a tight spot. However, if you come into work frequently and say –
“To meet our goals, we’ll have to work late again”, then you already have a problem that can’t be solved by working more hours”. (Beck, 2004)
So with constantly increasing scope but with a unrealistic fixed schedule, quality was being impacted. Corners that were cut due to unrealistic pressures imposed on the team and excessive overtime came back to haunt the team later. They were finding or making more bugs than before resulting in more delays and greater integration issues due to poor quality of work being provided by the offshore team.
Overtime just doesn’t yield the results promised by Gannt Charts which provide an oversimplified view of the software development process by only measuring the quantity of the hours spent on tasks without also taking into consideration the quality of the time (yes there is a difference).
Sometimes the biggest gains in productivity come from stopping to think about what you’re doing, why you’re doing it, and whether it’s a good idea. We need to work smarter, not harder and have the ability to hear unwelcome news and then address the real issues.
When trying to add new team members to address a tactical and fire-fighting need, we need to realize that an incremental person who, when added to a project makes it take more, not less time – nine women cannot make a baby in one month.
This recent Dilbert comic strip reminded me of a situation when a client was raising cosmetic defect as severity 2.
It is often not clear what a defect is and at what level to raise it at (despite agreed severity descriptions) as my example above shows. And some defects are raised without being related to a requirement. Or defects raised because of a feature being “not user friendly”.
Regardless of the reason and how you label the ‘defect’, I treat all defects the same – it is a piece of work required by the client that needs to be prioritized with other work. So the approach is to put all work (defect, enhancement, new feature etc) into the product backlog. As part of the planning process, the Product Owner will prioritize the items on the backlog for the next iteration and/or release. So if a cosmetic defect is more important (has greater business value) than a new feature, then it should be placed higher so it gets completed first.
Keeping all work in the product backlog, makes planning simpler and helps manage the flow of work. It also ensures the client is in control and be the judge of what is important to their business.
We sometimes hear that business is not getting value out of IT. Technology and tools are constantly improving but we often have missed expectations. The need to align business and IT is getting more important.
Incorrect assumptions of what the client needs often leads to incorrect actions resulting in wasted effort.
Next time you speak to your customer, clarify what they mean and speak in their language. Get the customer to provide real life examples and elicit feedback along the way. That way you will move towards aligning IT with the business and deliver greater value.
Source: Dilbert 21-Sep-2009
Dilbert cartoons are great because they poke humour in everyday working situations, and they wouldn’t be funny if they weren’t true. Too many times have I seen project teams in the situation above, including ones I have worked on.
Customers should be made part of the team. A goal in Agile is to have the customer collaborate with the team and a key role is the Product Owner. Together the team and customer work out what is needed in the user story (requirement), what is required to start the user story and what are the acceptance criteria to pass.
Communication between the customer and team occurs throughout the project and feedback is provided by the customer to the team often. By having the customer part of the team it helps the team maintain a state of productivity by remaining available to answer questions and further clarify acceptance criteria. Because the team and the customer work together to discuss the user story, there are no misunderstandings and situations in the above cartoon should rarely occur.