Monday, May 26, 2014

The Three Faces of Done

How can you expect me to test it if it's not done?!
I hear this question a lot from people just starting out with Scrum. What does it mean for something to be "Done" in Scrum?

Build the Right Thing, Build it Right, Build Enough

Under Scrum, the objective is to produce a potentially shippable product every sprint. Scrum is an incremental framework, which means you achieve your overall goal incrementally. Each sprint, even each backlog item, produces an additional slice of the product. This slice is integrated with all previous slices into a whole. That potentially shippable whole is called the "Increment."

Everything that goes into the Increment has to be "Done" and the increment as whole has to be "Done." If it's not done, you can't ship it. So what does it mean for something to be done? That's what the Definition of Done is for.

The Definition of Done is an agreement between all the members of the Scrum Team: what does it mean in your context for something to be done? So regardless of who you ask on whether something is done, you should always get the same answer.

To come up with a Definition of Done, I have found it useful to look at Doneness in the context of three questions:
  1. Have we built the right thing? (External Quality)
  2. Have we built it right? (Internal Quality)
  3. Have we built enough value to justify releasing? (Fitness for Use) 

External Quality -- Building the right thing. 

We could call this "Customer's Intent."

Imagine you are buying a car. You ask the car dealer for a red car with 2 doors, leather seats, automatic transmission, air conditioning, heated seats, and probably some other essential features. Your dealer can't deliver that car right away, so you come to an agreement to order that car, and 30 days later, that car is sitting in dealer's lot, waiting for you to pick it up. Did you get the car you ordered?

You check the car against the order: color, transmission, number of doors, presence of air conditioning and heated seats. If it matches the order, you say, "yes, this is the car I ordered." The car conforms to what you ordered, so you say thank you and take the car home.

Customer Intent, or building the right thing, is usually reflected in some kind of acceptance test. When this test passes, the customer is happy. For a car, we know in advance what a car is and what features it can have, so defining the car by a list of acceptance tests (aka - order form) is easy to do, and most customers can order a car at that level alone.

In Scrum, customer intent is identified during the conversations between the Development Team and the Product Owner or other domain experts. (This is often called Estimation, Backlog Refinement, or Release Planning.) The Product Backlog item is enriched with the "confirmation" - a description to confirm that the solution or implementation satisfies the objectives set for it. Scrum does not define how to do this. In a software development context, some widely used practices include: 
  • a simple workflow, "How To Demo," to show that the feature is working correctly.
  • a set of conditions and expected results ("Given <a known state> when <some action occurs> then <expect some result>"), or even 
  • a collection of examples that pass or fail. For example, to convert integers to Roman Numerals, you might have a table that starts like this:
    0 -> #badInputError
    1 -> i
    2 -> ii
    3 -> iii
    4-> iv
The latter two are examples of Specification by Example and can lead to your specifications becoming automated test suites.

Internal Quality / Building it right

We could call this "Developer's Intent."

But how do you know that the car is legal to operate in your jurisdiction? How do you know that the air conditioning will produce enough cold air so that you can drive comfortably in the heat of summer (without being so cold as to cause respiratory illness!) ? How do you even know it was built according to the designs of the engineers who designed it? How do you know that your new car satisfies the larger requirements to be road legal in your jurisdiction? How do you know that the car will outlive the warrantee?

So many questions! Can you verify this as a customer or even as a (non-technical) Product Owner? 

In general you are dependent on the manufacturer to do their job right. In Scrum, it's harder, because you are building a product for the first time. While the Development Team is responsible for achieving "Done" for each feature they build, the whole Scrum Team is responsible for the quality of the product.

How does the Development Team ensure internal quality? At this level, the Definition of Done looks like a checklist. Here is a typical Definition of Done for a moderately mature Scrum team that develops Software for an iPhone/iPad environment:
  1. Unit tests written
  2. Code checked in
  3. Code review completed
  4. Improvements from review implemented
  5. All new unit tests green
  6. All existing unit tests stay green 
  7. Build available for download in TestFlight
  8. Acceptance tests verified by Development Team
  9. Done confirmed by Product Owner
All these points ensure building it right. The last two points create a link between internal and external quality so you build the right thing. The Development Team verifies to the best of their ability that the "Customer Intent" has been satisfied. The Product Owner has final say over whether a feature is done or not.

This example includes automated testing. Why? If something was Done in the last sprint, shouldn't it stay Done in this sprint? If not, how can you call it done? 

Personally, I have had the best results applying the Definition of Done to each backlog item, not to the increment as a whole.

Unfinished work

When deciding whether a backlog item is done or not, the Product Owner should ask the Team, "Can we release it?" If the team says, "Nope, we can't release it" (or any of its more subtle variations), then the work is not Done, and the appropriate response is "Why not?" The answer is un-Done work. It goes into the product backlog -- because it has to be finished before you can release -- and should be a subject of the next retrospective: How can we have less un-Done work at the end of the next sprint?

Completion / Fitness for Use

Under what conditions do you actually ship the increment? This is a judgement call. The Definition of Done is not a Definition of Complete or Definition of Releasable for the product as a whole. Assuming there is no un-Done work, the Product Owner may decide to release at any time. The product should be releasable by the end of each Sprint, and possibly more often than that. Very mature teams in very dynamic environments often strive to achieve continuous deployment.

Under Scrum, you achieve releasability incrementally by implementing one feature at a time. Eventually you will have implemented enough features that the Product Owner decides that implementing more features will not add more value to the product and/or the cost of releasing is justified by the value to be earned by the release. In this case, the Product Owner asks for a release. If the Increment is truly done, the Team will respond, "Sure, no problem!"

Create a Sample Definition of Done - (Non-Software)

What if you are not doing software? How does this apply? Just ask your team:
  1. How do we ensure we have built the right thing? (External Quality)
  2. How do we ensure we built it right? (Internal Quality)
So let's say you are a consultancy and you produce studies. How would you answer these questions? What quality standards does a study have to fulfil in your organization? The answer might look something like this:
  • Does it conform to the appropriate CI/CD?
  • Does it contain customer CI/CD?
  • Has it been spell checked?
  • Has it been proof-read?
  • Has it been fact-checked?
  • Does it have enough pictures and graphs?
  • Have we confirmed that the content represents value to the customer?
  • Does the content address a key goal of the study?
If each backlog item corresponds to a (sub)chapter of the study, then it is pretty straightforward to apply these tests for each new chapter as you write them!

How done is Done?

When is something really done? Ultimately, something is only done when everyone has lost interest in it.

So almost by definition, your Definition of Done will only be partially done in the greater scheme of things. Does Done include User Acceptance Testing? Does it include Customer Acceptance Testing? Does it include validation from the marketplace that the feature is necessary and desirable? Is it feasible to check these things on a feature by feature basis?

In general, you should strive to make you Definition of Done more extensive as your project moves forward. As Done becomes "done-er" your quality is rising, and with it, your productivity.

If there is no alternative to handing your product off to a downstream process, then understanding what it means to be ready for that process can be very helpful. When I developed an iPad app, I had to be sure that Apple would accept it according to their terms and conditions. So my team looked at the contract, and we figured out Apple's Definition of Ready, and made sure that our product conformed. As a consequence, we are able to get our apps on to the app store without too much hassle.

If you have a downstream process, one question to raise is how can you integrate those stakeholders better, so that you are more in synch on what Done means and what is really done. If you have stakeholders or customers who also have to accept the product, how can you get them more involved in your process so that Customer Acceptance Testing happens during the project, not at the end, and any testing at the end becomes more a rubber stamp formality than source of risk and surprises?

If you are building for the market, you don't really know if you have built the right thing until you have gotten that confirmation from the market. How would you integrate market validation into your process?

Summary: The Three Faces of Done in Scrum

Scrum says simply, we have to agree what "Done" means. The definition of Done should help you answer the first two of these three questions effectively:
  1. Have we built the right thing? 
  2. Have we built it right?
  3. Have we built enough value to justify releasing?
The third question is a judgement call, and getting that right is the Product Owner's responsibility!

There is a bonus question, especially when developing software: How do we ensure that what gets Done in this Sprint, stays Done in future Sprints?

If your team can answer all these questions well, you are on your way to high and predictable team performance, delivery dates and product quality!

Credits: This perspective on Done and the terminology used was inspired the Lean Software Development series by Mary & Tom Poppendieck.

[Update: 11:00 I have tweaked this article a bit for spelling, grammar, readability and completeness. I need a Definition of Done for my blog articles ;-) ]

Tuesday, May 20, 2014

Bring back the fun! Four tips for the Product Owner...

...when the market isn't buying your product
"Is it OK to pass on market pressure to my team?"
This question came up in my last Certified Scrum Product Owner course ("Leading Innovation").  It seems that the company's development efforts, i.e. their products, had not been producing the desired effects in the marketplace. As a result, the finances were not looking so rosy, the investors were getting worried, budget cuts were in the air, and who knows what that means for people's jobs! The PO (recently converted project managers) wanted to put the teams under more pressure to perform, feeling it was both necessary and justified. Scrum says the teams are protected from undue influence, so we have a typical conflict between the classical approach and the Scrum approach.

So let's look at the this pressure, where it comes from, what to do about it, and how to best react to this situations.

Where is the pressure coming from?

Most of the time, the market doesn't care about us. We make something, nobody notices. So we compensate: with spam, advertising, marketing events, cold calls, etc. We create outward pressure, and customers should notice us and buy our products. If the customer is not buying, it is because we are not pushing hard enough. Why are we pushing? Because the market does not care.

So the pressure is not from the market, but from ourselves, because our lack of success will sooner or later cause us problems.

This does not mean the market does not exert any pressure. It can. It is called demand. It is the opposite of pushing. Pull is created when the market (or more precisely, people outside our company) genuinely want our product. Think people standing in line to buy the latest iPhone or see the latest blockbuster.

The first step is to realize that, unless the market is sucking product out of us faster than we can make it, the pressure is internal and entirely under our control. The trick therefore is to create pull for our products, so that there is a ready market.

How do we create pull?

In my eyes, the messages of Lean Startup from Eric Ries and Ash Maury give excellent guidance: find a problem that resonates with your (potential) customer base. Validate that it really is a problem in the eyes of your potential customers. Create a enough of solution so you can validate the solution with your potential customers. Validate it. Get your customers to tell you what it is really worth (by framing it against other existing solutions). If you get these things right, your customers will eat out of your hands! Note: This process may many iterations and require several changes to the initial product idea!

How should you lead your team?

To create products you need to be creative. Dan Pink explained the prerequisites: Automony, Mastery and Purpose. He also told us about the anti-pattern: things that cause to you to focus too much on the goal, like fear and financial incentives.

Most of us have had a "best project", where we did something great for our customers, our colleagues were helping us, as developers we were writing great code, our managers had our backs, etc. Remember that project? What would it be like if this project because like your best project. You can make it happen! (Hint: Many people using Scrum are experiencing their best project right now!). So make this project your best project for everybody on your team!

What to do when the market isn't buying your product

Your mission is to reduce risk. At the beginning of product development, Market Risk and Social Risk usually represent the biggest challenges. Are you building the right thing? Can your team work together to build the product? Budget Risk may also be an issue if your funds are limited. How can you get happy outcomes for your customers with less expenditures from your side?

As Product Owner, what can you do to minimize these risks when the market isn't buying your product? Four suggestions:
  1. Social Risk 1: Get any bad news out of the way. If there will be layoffs, get them done as quickly and as painlessly as possible. Once they are done, ensure that your team can continue long enough to show results.
  2. Social Risk 2: Bring back the fun! Create an environment where you and your team can have fun and serve a great purpose. Motivation is the accelerator for productivity. 
  3. Market Risk: Get you and your team close to your customers. Figure out where the shoe hurts. Offer them solutions. Validate your assumptions!
  4. Budget Risk: Say no. Don't build something unless you are really certain your customers need it and will value it!

Update: 21-Mai-2014 - after speaking with another startup who was having the same problem, I updated this article to emphasize the fun aspect!

Monday, May 19, 2014

Three things to like about SAFe

I have written skeptically about SAFe and other approaches to scaling Agile. Some people have written even more skeptically (see Daniel GulloRon JeffriesDavid Anderson, Ken Schwaber and Dave Snowden for examples). Last weekend, I was able to take a closer look at SAFe through the eyes of its practitioners, and to my surprise, I found three things to like about it.

The Swiss Agile Coach Coach Camp in Kandersteg was an amazing opportunity for top practitioners to learn from each other. I asked Matthew Caine, who had been doing some SAFe implementations, if he would take half an hour or so to explain the key principles, to which he readily agreed.

I really appreciated Matthew's candor on SAFe's purpose and limitations. SAFe is not for corporate IT environments. It is for (parts of) companies that produce big software products. If your organization is already agile, it will slow you down. So if you are able to deliver functionality at least every two months, SAFe will not help you. (A coach from another company confirmed that they said no to SAFe for exactly that reason). But if your company is challenged to release once or twice a year, SAFe could be a good thing for you.

After listening to Matthew and other practitioners explain how SaFE works, both on paper and how they have actually gone about implementing it, I have found three things to like about SaFE:

  1. It has Scrum and Kanban inside. Yes, SAFe redefines both of them in ways that their respective communities are not happy about (see above links). But it legitimizes their presence, makes them part of the system, and leaves the door open to further improvements later.
  2. It defines work for all three levels of the organization: Top (Portfolio) Management, whose job it is to decide where to spend the company's money. Mid-level (Program Management), whose job it is to define the functional objectives of each quarterly release (the "release train" or "release choo-choo" as it is affectionately known), and the operational level, which creates stories, refines the backlog, and implements the functionality.
  3. SaFE can be implemented incrementally. It need not be a big bang migration. However, it is implemented in vertical slices of the company, from top to bottom, and can be applied to "thin slices" which simplifies the implementation tremendously. So you start with all levels of the organization associated with one particular product, implement SAFe in that slice, and assuming it is successful, expand to other slices of the organization incrementally. 
It was points two and three that really got my attention. 

When scaling Scrum, we often talk about feature teams (vertical slices from the front end to the back-end) vs. component teams (horizontal slices for each functional layer). Conway's law suggests that feature teams are more effective, and most coaches follow this approach. 

Scrum creates highly effective teams, but does not provide any guidance for the rest of the organization. This is both a strength, because it enables productivity without needing buy-in from the rest of the organization, but it is also a weakness. Since it doesn't need buy-in from the rest of the business, adoption is often challenged by the pre-existing organizational barriers between management and operational teams. Each agile coach has to find his formula to get buy-in from the organization.

So SAFe represents an approach to bring in Agility to the top of the organization and implement it at all levels. I have raised my likelihood to recommend score from a 4 to a 6.  I still have my reservations about it's lack of commitment to Agile values and I also have reservations about top-down change processes. But this does not prevent SAFe's practitioners from sharing the Agile values or applying good change leadership practices.

Jesper Boeg recently gave a talk "My Agile Journey: XP, Scrum, Kanban and Back Again" in which he said an agile transition was like mountain climbing. The summit is a high performance team. Scrum and Kanban represent base camps on the way. 

If we say our larger purpose is to create a high performance organization, then SAFe might represent a base camp along the way. Perhaps a better analogy would be that it represents the helicopter to get you to the base camp. So it looks like that helicopter will get you to 2000m, but not to 4000m. If you left some baggage behind, you could get higher. Does that make it a bad helicopter? 

Monday, May 5, 2014

How do you improve the waterfall with Scrum or Agile?

Many Project Managers come to my courses to find out whether Scrum and Agile are something for them and their for organizations. Sometimes the answer regarding Scrum is 'No.' What then? What can you do with the lessons of Scrum to improve your productivity if you're constrained to phase driven development? This is where the values of the Agile Manifesto can be really helpful!

Yvonne Horst, a Project Leader for Swisscom and recent CSM, just sent me a nice example (she also sent a nice recommendation for my course, but I digress):
"I would like to point out one idea. The usual staffing approach for projects in a non-Scrum environment is to send specialists into a project and have them produce the artifacts of the respective phase. At the end of the phase the artifacts are handed over to the next group of specialists, which makes it sometimes extremely difficult to ensure the know-how transfer over the whole lifespan of the project.

"The idea of the Scrum team with all necessary skills on board to solve whatever problem that may occur can be transferred into a waterfall team as well – if only by booking one or two key resources over the whole lifespan of the project.

"I want to go one step further now and am currently building a team of requirements engineering specialists, each and every one of them with in depth know-how of at least one core system. They will no longer be sent to projects, but the projects may in future obtain results from this team for their specific problem/changes/challenges/high level requirements with reference to the represented core systems. The team works and acts together.

"The first feedback from the business units as well as the Project Managers has been enthusiastic. And the team members tell me that they have learned more from their colleagues in the last two weeks than in the two years before…"
Here we have an example of applying both a Scrum practice and the Agile value, "People and Interactions". You can apply the Manifesto for Agile Software Development to any context. You do not have to do Scrum or XP to be Agile! Just value the "stuff on the left" more in your decision making:
  • Individuals and interactions
  • Working software
  • Customer collaboration
  • Responding to change
What are you already doing that values Individuals and Interactions or Working Software? What could you do more of (or less of) to emphasize these values? What could you start doing?

And Scrum itself can provide the inspiration for possible improvements. How could you apply timeboxing? How could you inspect and adapt more frequently on the basis of honest information?

I am sure there are other examples of applying Agile Values and Scrum Practices to traditional project management. I'd love to hear your suggestions and examples!