Monday, March 14, 2011

Success Factors in a Scrum Sprint

Last week, I posted a summary of the responses to my poll on scrum sprint success and story size. I also proposed a simple definition of success: a successful Scrum team delivers what it promises.

A good Scrum team does not systematically over-commit. It doesn't under-commit either - not one single response indicated that the team usually delivered more that 110% of its commitment. By this definition, 32% of the respondents are "successful" while 68% are "improvable".

Let's take a closer look at Team Size, Story Size and Sprint Duration and how the relate to relate to successfully completing the a Scrum Sprint.

Team Size and Sprint Duration

Two week sprints are by far the most popular with 50 respondents, and the distribution of on the number of stories committed to each sprint looks pretty much like a bell curve centered around 6 stories/sprints. Three week sprints a relatively popular - 17 responses - but the volume of data is probably too small for a curve to form.  With a total of 13 responses, it's unlikely that the data for 1-week and 4-week sprints is at all representative.

Number of Stories

The number of committed stories peaks at 6 to 8 for teams doing two-week sprints and 4 to 5 for teams doing 3-week sprints. I am puzzled by the dip at 6 to 8 stories for three week sprints.

Number of Story and Sprint Success 

Queuing theory tells us that implementing more, smaller stories should be more predictable than fewer large stories. The numbers generally bear this out. (The data point for 2 to 3 stories is probably not representative - there were only 5 responses, all others had 15 or more responses).

Sprint Length and Team Success

Are shorter sprints better? A lot of common wisdom points to this conclusion, but it is not supported by the poll data. The average of all the data was that 38% of the teams are 'successful.' Teams doing 2 and 3 week sprints reported slightly worse than average and 1-week teams slightly better.

Most intriguing are the teams who reported 4 weeks sprints. They reported a success rate of 75%! There are only 8 teams reporting, so it may not be representative. But it may also be that the value of longer sprints are underestimated.

Team Size and Sprint Success

Teams with 1 to 3, 6 or 9 to 10 remembers were less successful that the other sizes. I don't think there is much of a trend here.

Maximum Story Size and Sprint Success

I was expecting that teams which took on very large stories would be less successful than those which do not. This was only true for those teams which reported taking on stories larger than 40% of the teams capacity for the sprint.  11 Teams reported taking on stories larger than 40% of the team capacity, only one reported being usually successful in accomplishing the sprint.

Much to my surprise, teams reporting taking on 35 to 40% were the most successful, followed by teams that limit the largest story to 5%. The sample sizes for those data points were rather small, with 5 or 6 respondents in each category, so I would take that with a grain of salt. Among those categories with more than 15 responses, the range of 15 to 30% did best.


Having analyzed the data, I think the survey would be much useful if there were a factor of 10 more data points - 800 to 1000 would be really good. Still, looking at this data, some trends are taking shape:
  • Doing Scrum well is very difficult. Most teams do not reliably deliver their sprint commitment by the end of the sprint.
  • There is no one right way to do Scrum. Both "Successful" and "Improvable" teams come in all sizes and configurations.
  • At least up to a point, more stories per sprint are better. It looks like 10 to 15 is a good number. Most teams should consider slicing their stories smaller.
  • Very big stories are are unlikely to be completed in the sprint, but an occasional moderately sized story doesn't look like it's a problem.
  • Maybe 4 weeks sprints are better than their reputation. The results could be wrong, but should be explored more deeply.
P.S. Does anybody with substantial readership think this study was cool and useful? I would love to repeat it with the goal of substantially higher feedback and see the trends become more apparent. Please contact me!

The story size poll is still active - I'll keep an I eye on it and when I get 200 responses, I'll evaluate the data again. Presently I have 85.


Kurt Häusler said...

Very insightful, thanks a lot!

Scrum Software said...

Very interesting statistics here.. I have to agree that there is no one way to develop with scrum. Companies and Project manager's sometimes seem to get caught up in all of the rules/fads of scrum instead of creating a system that works with their culture and team. Excellent post and I look forward to reading more.

Coach at Play said...

I think this was very interesting as well. It would be nice to try it again and push for more input to see if the trends become a little bit more clear and then maybe ask some clarifying questions to try and draw unique differences and similarities.