Skip to main content

Quick Poll: A Litmus Test for Agile Development

This summer, I asked how many teams are doing Scrum according to the minimal definitions of the Nokia Test. 74% of Scrum Teams responding are doing what Jeff Sutherland now calls Scrum-Butt - "we're doing Scrum, but for some reason, we can't do all of Scrum." According to Jeff, even Scrum-Butt companies may improve their revenue, but those who go beyond Scrum-Butt do much better financially than those who accept dysfunctions. Jeff has extended the Nokia test to identify the factors which help companies achieve this "hyper-productive" state.

Most of my clients are not yet ready to extend the envelope. They are trying to achieve the basics of good agile management and development. For these companies, the Nokia test is a good place to start, an early milestone, but not the final goal. (BTW - Alistair Cockburn's 7 Crystal Properties also look like a good starting point, and some of his points are raised in the candidate list below).

The next question is how is software engineering doing? I want a litmus test, i.e. a short list of questions for challenging developers and their management on their engineering practices.

My question to you: What questions make a litmus test for "pretty good agile development?" My goal is to come up with ten to fifteen yes/no questions.

The Joel Test was an early example, but is now dated. There have been several attempts at more agile definitions of the test (e.g. confused of calcutta and jbrister), but these have not been validated. All of these lists contributed to the list of candidates, below.

This week, I ask for your help in picking the questions. Next week (or so), I will summarize and then conduct a survey based the questions you select.

Here is the poll: Which questions make up the Litmus Test for Pretty Good Agile Development?
  1. Source Code: Do you use source control?
  2. Build: Can you build in a single step?
  3. Daily Build: Do you make daily builds?
  4. Bug DB: Do you have a bug database?
  5. Fixing: Do you fix bugs before writing new code?
  6. Sched: Do you have an up-to-date schedule?
  7. Spec: Do you have a spec?
  8. Quiet: Do programmers have quiet working conditions?
  9. Tools: Do you use the best tools money can buy?
  10. Testers: Do you have testers?
  11. Intervew: Do new candidates write code during their interview?
  12. Hallway: Do you do hallway usability testing?
  13. Wiki: Do you use a Wiki?
  14. Continuous: Do you do continuous build / test / deploy?
  15. TDDev: Do your tests drive your development?
  16. Pair: Do your developers pair and support each other?
  17. Talk: Does everyone talk to each other, constantly?
  18. Hiring: Does the team select its new members?
  19. Colocated: Is the team colocated?
  20. Testing: Can you test in a single step?
  21. Releases: Have you delivered running, tested, usable functionality to users at least twice in the last six months?
  22. Deploy: Can you deploy in a single step?
  23. Integration: Do you integrate the system at least twice per week`
  24. News: Can you give your boss bad news?
  25. Access: Does it take less than three days from when you have a question to when an expert answers it?
  26. Improvement: Did you get together within the last three months to discuss and improve your group’s working habits?
  27. Retrospective: Does you team conduct a retrospective after every iteration?
  28. User Stories: Do you define the product in terms of user stories?
  29. Acceptence: Do you define acceptence tests before you write code?
BTW 1:All questions are in the form key-word:question. The keyword is there to help readability of the poll in doodle.

BTW 2: I will be at the Scrum Gathering in Stockholm and look forward to meeting as many of my readers as possible! Please let me know if you're coming!

BTW 3: Voting closes Midnight (UTC) on October 25. So vote now!

Have I missed anything important? That's what comments are for ;-)

Thanks for your help!


Unknown said…
Knowing whether my team is agile is less useful than knowing if we are providing value to our customers. But if the test is structured in such a way that the questions can help identify behavior vectors which might help us improve, then that would be OK. Given that:

In this test is "quiet environment" a good thing or a bad thing? When I'm in our team area and it's too quiet, then I know communication isn't going on and that's generally not a good thing.

Does "colocated" mean cubes in the same area, or an open team work area where team members can easily move around to help each other, pair on work, and rapidly gather around the whiteboard to discuss today's design question? Cubes in the same area is better than different buildings or cities, an open reconfigurable work area is much better yet.

Does "Can you test in a single step?" mean "Are the vast majority of your system tests automated?"? If not, what does it mean?

Testers: Is having testers a good thing? I personally believe so. I think the question should focus on how the testers and developers work together to deliver the best possible system: Daily collaboration, joint creation of test automation tools, etc.

Releases: I think twice in six months is an incredibly low bar for agility.

Wiki: While I encourage teams to use a wiki for both near-time communication and as a simple method for knowledge capture, the question seems too focussed on a technology. I'm sure there are teams who use Sharepoint or some other technology in the same manner we use a wiki.

Proposed additional questions:

Do you create an automated unit test to replicate the problem before fixing any issue discovered by testers or in production?

Are there areas of the system that only one person can work on?
Anonymous said…
I don't like the test. I don't think it will be valuable and not sure what goals it have behind.
Peter said…

Thanks for your feedback!

yes, a lot of questions are open for interpretation. In the original Joel test, "quiet environment" probably meant "developers have their own offices, can close the door and be left alone." Today it would probably mean the teams have a team space where they can talk as they see fit, but which is free of ambient noise from other groups, construction, etc. And probably the team members would like access to quiet rooms, so they can close the door and concentrate when they need to.

Peter said…
Hi Anonymous,

I am sorry you feel that way. But let me explain why I am asking these questions.

I often go in to projects that are "challenged." They are not delivering to management expectations. The project is my patient, I'm their doctor. I want to take its pulse, check its temperature, look its tongue, whatever, to come up with a first diagnosis of what the of what is ailing the project, so that I can help it (or more precisely, the team) back to "health".

This is perhaps a first iteration. I have had some discussion about this at the Stockholm Scrum Gathering, and one very good idea, would be to come up with a list of "Smells" (perhaps symptoms would be a better name) and their causes.

Any ideas?



Popular posts from this blog

Sample Definition of Done

Why does Scrum have a Definition of Done? Simple, everyone involved in the project needs to know and understand what Done means. Furthermore, Done should be really done, as in, 'there is nothing stopping us from earning value with this function, except maybe the go-ahead from the Product Owner. Consider the alternative:
Project Manager: Is this function done?
Developer: Yes
Project Manager: So we can ship it?
Developer: Well, No. It needs to be tested, and I need to write some documentation, but the code works, really. I tested it... (pause) ...on my machine. What's wrong with this exchange? To the developer and to the project manager, "done" means something rather different. To the developer in this case, done means: "I don't have to work on this piece of code any more (unless the tester tells me something is wrong)." The project leader is looking for a statement that the code is ready to ship.

At its most basic level, a definition of Done creates a sh…

Explaining Story Points to Management

During the February Scrum Breakfast in Zurich, the question arised, "How do I explain Story Points to Management?" A good question, and in all honesty, developers can be an even more critical audience than managers.

Traditional estimates attempt to answer the question, "how long will it take to develop X?" I could ask you a similar question, "How long does it take to get the nearest train station?

The answer, measured in time, depends on two things, the distance and the speed. Depending on whether I plan to go by car, by foot, by bicycle or (my personal favorite for short distances) trottinette, the answer can vary dramatically. So it is with software development. The productivity of a developer can vary dramatically, both as a function of innate ability and whether the task at hand plays to his strong points, so the time to produce a piece of software can vary dramatically. But the complexity of the problem doesn't depend on the person solving it, just …

Money for Nothing, Changes for Free

“Money for Nothing, Changes for Free” encourages both customers and suppliers to focus on value.

A key advantage of Scrum projects is that at least once per sprint you have something that could be shipped and no work in progress. You can change direction every sprint, and you can reevaluate whether the project is a good investment or if your money could be better spent elsewhere. Abrupt cancellation is risky for the supplier.

While the concept of an early-exit penalty is not new, Jeff Sutherland gave it a unique allure with his allusion to the Dire Straits hit.
Desired Benefit Incentivize both customers and suppliers to focus on functionality that provides genuine value.
Structure This works with Agile software projects because there is little or no work in progress. After each Sprint, functionality is either complete or not started. Work is basically on a Time and Materials basis with a cost target, often with the intention that the project should not use up the entire project budge…