Agile Scientific Research?

We're doing research. We don't have a customer telling us what to build. Can we do Scrum? What do we put in the backlog?

Sometimes questions get asked so often, that they just demand an answer! When a research group at the EPFL asked me to come and teach them Scrum, they also represented the fourth time this year that someone had asked me these questions!

Before we try to answer them, let's take a look at whether scientific research is similar enough to software development for the values and principles of the Agile Manifesto to really apply!

Science is Different, but Similar

In a commercial environment, there is usually someone who understands what needs to be done. In Science, figuring out how things really work -- and excluding other ways that things might work but don't -- is the the core business. What they have in common is complex problem solving. The feedback comes from users in case of software development and nature in the case of science (though much science involves software development).

Often scientific problem solving is so complex that only a very small number of people understand the underlying models and phenomena. This makes it difficult to simply delegate programming assignments. OTOH the scientists themselves are usually not good software developers. Sponsors of research come with a question they want answered, not a list of features to implement. Trying to select the optimal form of collaboration between scientists, software developers and other stakeholders raises a lot of questions.

The first special characteristic I noticed about the scientific environment was a strong "yes-but" culture. Unlike many commercial environments I have seen, the yes-but culture was not perceived as being toxic, though it did seem to cause some of the expected difficulties in cooperation -- or should I say, lack thereof? Still, challenging assumptions is essential to science. If you observe B, and want to show that A causes B, you have to exclude every other possible cause of B, before you can really conclude that A is the cause. Yes-but is a professional skill.

At the same time, developing an idea often requires a constructive yes-and mentality. Yes-but tends to kill ideas and to make collaboration difficult. So both approaches, yes-but and yes-but, have their place in Science. I am wondering if scientists were to be yes-but and yes-and into their hypothetical Manifesto for Agile Scientific Research, which one would be on the left?

Is the Manifesto for Agile Software Development Relevant to Science?

"We are uncovering better ways of developing software, by doing it and by helping others to do it...." Scientists are not developing software, nor do they do research for customers (per se), so how much of the Manifesto is really relevant?

Suppose we make a few small changes to that manifesto? I used the following modified version to lead some thought exercises during the Scrum Training:

Draft: Manifesto for Agile Scientific Research

We are uncovering better ways of performing Scientific Research by doing it and helping others do it.

Through this work we have come to value:

• Individuals and interactions over processes and tools
• Scientific discoveries over comprehensive documentation
• Collaboration over contract negotiation
• Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

(with apologies to Alistair Cockburn and company!)

This is the approach that the devops movement has taken, modifying the Manifesto to identify more directly with the challenges of running systems. But will it work for Science?

Step one is to try it on for size. Look for some practices that feel right or feel wrong. (I will skip the positive examples. Most companies that try this exercise find that the decisions that value stuff on the left are generally better both for their customers and for themselves than those decisions that value stuff on the right.)

Publish, Perish or Discover?

"Publish or perish" is a well known fact of life in academic circles, and it is even more true in the Scientific community. To succeed as a scientist, you need to get published as the lead author (with a bunch of minions supporting you!) in an important journal. But what if your research is too big for one person? What if you're part of team? How can you reconcile working in a team when your career requires you to shine as an individual?

This proved to be a hot topic in the scientific community! Some scientists even questioned whether scientific discoveries and the published paper aren't the same thing!

Publish or perish is something of a fact of life, but it is in conflict with the needs of complex research -- think CERN with its billion-dollar, multi-decade research mandate. How many lead researchers can there be? What is the interest of a researcher to be a team player in a complex field, if the industry does reward such efforts?

This draft manifesto suggests that better ways of doing research will value discoveries over individual glory and that changes to the current rules of the game will be good for future discoveries!

Towards a Culture of Continuous Improvement

I led my class through an exercise of identifying cases where the institute already values things on the left. These precedence cases confirm the validity of an Agile approach.  Then they identified potential areas of improvement the would come by valuing stuff on the left more.

Then they prioritized these potential improvements to identify what would help the organization most. This produced a backlog for starting a continuous improvement in the research process. The most exciting part was watching the light go in in people's eyes when they saw how this process, if implemented consequently, could turn any organization into an awesome place!

Agile Scientific Research?

Being Agile doesn't mean simply doing Scrum (although Scrum may well prove useful in many scientific contexts). The juxtaposition of things that we value, e.g. discoveries and publishing, provides the basis for some fascinating discussions, which are resonating with the deeply important issues of the wider community.

The Manifesto for Agile Software Development is powerful tool for understanding the priorities in an organization or industry, determining whether you are really setting the right priorities, and for identifying improvements to make the organization (or even the industry) more effective. It naturally leads an organization to a culture of servant leadership. With a little bit of modification, it can serve many different human endeavors, including scientific research.

I hope that the principals involved here will remain intrigued by what they learned in their Scrum course and start to build a community beyond their institutional boundaries to look for better ways of performing scientific research, by doing it themselves, and by helping others to do it.

12 Tips for Product Owners who want better performance from their Scrum Teams

As Product Owner, I want better performance from my Scrum team.

Jeff Sutherland observed that teams in the 95th percentile were 10 times more productive than average teams (50th percentile). He created Scrum to enable all teams to recreate those conditions that enable top performance.

How do you really get better performance from your Scrum team? It’s not about cracking the whip. That is usually counterproductive. It’s about creating the right conditions. In my Product Owner course, we look at the many tools available to Product Owner to enable better and quicker results from the development teams. On the saat-network company site, I have posted 12 tips (well, 13 actually) for how Product Owners can improve Scrum Team performance. Read the whole article...

Solving the Social Media Challenge

Delight the customer. Inspect and adapt. I've been teaching that for years. And I have been working on eebee, a feedback solution, to help anyone who holds events to get more and better feedback from their participants. 

Then I got an email from a hotel director I stayed at. "If you liked us, click here to tell TripAdvisor. If not, click here to tell me." 

Boy, what a decision! He has no idea if I'm happy or not, but an immortal comment on the Social Media is just one click away! Feedback is about more than listening to the customer, it's about guiding the conversation, before it goes viral! Steve Holyer and I made a video, to explain the problem:

Does this resonate with you?

Weekend ScrumMaster Course?

Every once in a while, I talk to a consulting company who says, "we really want to do Scrum, but our people work during the week (by the hour). We can't take them out of a customer project."

OTOH I am told (have experienced, and actually enjoy the fact that) "in Switzerland, we don't work on the weekends." 

Interesting conflict.

So here is an experiment and you can influence whether it succeeds: This June 14 and 15, I will hold a Weekend CSM course in Zurich, AFAIK the first. That's a Saturday and Sunday. We'll start earlier than usual, so you can finish in time to enjoy Saturday evening activities. 

What do you think of the idea? I'd love to hear your comments!

Do you know someone, who can't take a Scrum course during the week? Have them check out "Weekend CSM"  

Scaling Scrum: SAFe, DAD, or LeSS?

Participants in last week's Scrum MasterClass wanted to evaluate approaches to scaling Scrum and Agile for their large enterprise. So I set out to review the available frameworks. Which one is best for your situation?

Recently a number of approaches have started gaining attention, including the Scaled Agile Framework ("SAFe") by Dean Leffingwell, Disciplined Agile Development (DAD), by Scott Ambler, and Large Scale Scrum (LeSS), by Craig Larman and Bas Vodde. (Follow the links for white papers or overviews of each approach).

How to compare these approaches? My starting point is Scrum in the team. Scrum has proven very effective at helping teams perform, even though it does not directly address the issues surrounding larger organizations and teams. An approach to scaling Scrum should not be inconsistent with Scrum itself.

Scrum implements a small number of principles and constraints: Inspect and Adapt. An interdisciplinary Team solves the problem. Deliver something of value at least every 30 days. One voice speaks for the customer. A coach and change agent helps the team and the organization improve.

Scrum also creates a context where the Agile values and principles can thrive. "We are uncovering better ways of developing software by doing it and helping others.... We value individuals and interactions over processes and tools.... Our highest priority is to satisfy the customer through early and continuous delivery of valuable software...." As anyone who has attended my Agile company workshop knows, if you use the values of the Agile Manifesto as a guide, you generally make pretty good decisions about how to run your company.

So how do SAFe, DAD and LeSS stack up for scaling Scrum and Agile? I will look briefly at each one, by examining what it claims to be, what its core values are, and give an assessment of how its values and principles correspond to those of Scrum and Agile. For reference, let's start with the Waterfall.

Classical Management or "Waterfall"

IMHO, the waterfall is simply adapting the management structures created to manage the automobile industry in the early 20th century to software. Assembly lines assemble components in steps defined by the engineers and managers who watch over unskilled and recalcitrant workers. Companies seek to maximize outputs and profits, and minimize costs. According to Steve Denning, the underlying principles are managers as controllers, coordination through bureaucracy, optimizing utilization, and communication primarily as a from-the-top broadcast.

This approach worked extremely well until about the 1960's or 1970's. Since then it has come increasingly under strain. Deloitte's Center for the Edge predicts that 70% of today's Fortune 1000 companies will not be there 10 years from now. This model doesn't work any more.

Agile and Scrum represent a rejection of this approach for the simple reason that it fails spectacularly when applied to producing software. This also makes clear why scaling Scrum is such a challenge for organizations. It is based on values and principles that are incompatible with the rest of the organization.

Summary: On a 0 to 10 scale, how likely am I to recommend this approach to scaling Scrum? 0.

It's failing. Time to move on. (And yes, I know, most every company on the face of the earth is organized this way. But look at the exceptions: Spotify, Guidewire, Morning Star, WL Gore. There are better ways to organize your company.)

Scaled Agile Framework (SAFe)

The Scaled Agile Framework has been getting a lot of attention lately. There is a budding certification program, a roadmap for implementation (with lots of training), and support from various established companies (like Rally). As I write this, the SAFe Academy is listing 5 trainers and claiming over 1000 certifications have been earned at their trainings.

SAFe is "an interactive knowledge base for implementing agile practices at enterprise scale." The SAFe big picture addresses the enterprise at three levels: Team, Program, and Portfolio. At the Team level, SAFe looks a lot like Scrum, including of course XP practices. Not every sprint necessarily produces a potentially shippable increment, but this should happen frequently, possibly after a hardening sprint.

At the Program Level, the efforts the Agile teams are aligned and integrated to serve the needs of the enterprise and its stakeholders. SAFe provides a fair amount of detail on how to do this. The Portfolio level provides similar product and goal alignment between the investment levels and the operational levels of the organization.

What does SAFe value? According to the Core Values page, "we know that Lean thinking, the Principles of Product Development Flow and the extensive benefits that Agile development (Agile Manifesto, Scrum, XP technical practices, Kanban) all play important roles in defining the principles and practices of SAFe," but SAFe "really values" Alignment, Code Quality, Transparency and Program Execution.

This is the statement that worries me about SAFe. The values, principles, and practices that enable Scrum, Agile, and Kanban to work their magic, are important, yada, yada, but subjugated to "what we really value." There is little reason to challenge Code Quality and Transparency, but Alignment and Program Execution mean the development teams are expected to do what top management tells them; the difficulty that "mere mortals" have convincing top management to pursue new innovations is a well documented cause of large organizations' vulnerability to disruptive innovation.

Summary: On a 0 to 10 scale, how likely am I to recommend SAFe for scaling Scrum? 4.

It's better than waterfall, and existing managers will feel at home. But its lack of commitment to Agile Values, especially continuous learning and people over process, means that there will still be managers making decisions over the heads of people who understand the problem better than they do. This does not bode well for the acceptance in the trenches of SAFe and it will be interesting to see how many of SAFe's reference customers are still boasting about it 5 years from now.

Disciplined Agile Development (DAD)

After I got over the huge IBM logo on the cover page, I found quite a bit to like about DAD. This process framework is "a people-first, learning-oriented hybrid agile approach to IT solution delivery. It has a risk-value life cycle, is goal-driven, and is enterprise aware." What does DAD value? It's top four priorities are: 1) People first 2) Learning-oriented 3) Agile and 4) Hybrid.

Hybrid means that DAD also draws on other, more traditional sources, especially the various flavors of Unified Process for governance and life-cycle management. Projects are divided into three phases, Inception, Construction and Transition. Compared to Scrum, DAD puts more emphasis on architecture and technical risk reduction through the designation of an Architecture Owner. (DAD also changes many of the names of Scrum, so the Scrum Master is now the "Team Lead").

Back in 2008, I wrote that people and responsibility were missing from the Rational Unified Process. This latest evolution is clearly a huge step in the right direction.  The notion of "Potentially Consumable Service" as opposed to "Potentially Shippable Product" is intriguing.  OTOH, our understanding of risk has grown substantially since the days of RUP to include Market Risk and Social Risk, among others. Furthermore, this approach does not look very close to the way successful startups (Facebook, Spotify, Guidewire) are actually scaling up.

DAD now has a certification program. As of this writing, there are 9 trainers and 29 certified DADs.

Summary: On a 0 to 10 scale, how likely am I to recommend DAD for scaling Scrum? 7.

Much good stuff, and nothing obviously evil. My own experience both with technical risks and chief anythings (developer, architect, project leader, etc.) lead me to be skeptical about those aspects, but not so skeptical to throw the baby out with the bath.

I almost gave DAD an 8, but I lowered the score for pitching IBM tools in the white paper (compare to LeSS which encourages Open Source).

Large Scale Scrum (LeSS)

"Large-scale Scrum is regular Scrum applied to large-scale development." Craig Larman and Bas Vodde set out to manage large projects while staying within the constraints of vanilla Scrum. They have developed two frameworks depending on the size of the project. Because they remain true to the constraints for Scrum, Large Scale Scrum cannot be considered a practice. It is an organizational design framework.

Framework-1 is for projects of up to around 10 teams. The basic roles are unchanged, but some the of the meetings are changed and some are replicated at the-cross team level. For example, Sprint Planning 1 may be held with representatives of each team, rather than all members of all teams. Similarly, a cross team retrospective with representatives of each team facilitates overall improvement. Teams are organized as Feature-Teams. Other inter-team coordination meetings may be added, in the form of Scrum of Scrums or Open Space meetings.

Framework-2 is designed for even larger projects. Framework-2 adds an additional role, the Area Product Owner, who assumes product Ownership of a major section of the product. At this point, an Overall Sprint Review and Retrospective is also added to ensure overall product consistency and process improvement.

Beyond Scrum, there are many technical practices which are helpful and encouraged to enhance coordination: Continuous Integration. Internal Open Source (any source can be modified by anyone), Team-controlled build systems. These become even more important for multi-site projects. Pervasive video, Free Web 2.0 tools (skype, google docs) and open source tooling reduce the friction between teams.

Summary: On a 0 to 10 scale, how likely am I to recommend LeSS for scaling Scrum? 9.

One the plus side, LeSS is clearly in the Scrum and Agile tradition. It is the simplest of the three approaches and makes only a few changes to vanilla Scrum. When I look at Spotify, an organization that has scaled from 6 to 1200 staff members, I see a company architecture that is very close to LeSS. It will be a very natural approach for small organizations that are scaling up as they grow.

As with any Scrum transition, moving to LeSS is a complete architecture-redesign. Informed buy-in is the key to acceptance. I can imagine in larger companies, that could be a challenge.

Overall Summary

How likely am I to recommend these approaches?

• Waterfall: 0
• SAFe: 4
• DAD: 7
• LeSS: 9

Of these approaches, Waterfall has by far the best training infrastructure. Every year, Business Schools around the world churn out thousands of MBA's who know how to manage by the numbers, but have never heard of Agile, much less thought about Agile forms of management.

SAFe comes in a close second. With its training academy, licensed trainers, and traditional management-friendly approach, it is clearly resonating well and is poised to do well in the market. (The reports I have gotten on the quality of the courses is strongly mixed. Some say horrible, others seem to be eating it up).

Will SAFe be enough of break with the waterfall to save their customers from being part of the 70% that won't be in the Fortune 1000 ten years from now? Fundamentally it's people separated by two or three layers of management from the real world setting direction for those who get it. Not promising.

On paper, DAD has come a long way since RUP. It's values are more in sync with Agile than ever. Today, DAD has more trainers than SAFe, but hardly anyone has actually taken the course, so is it really resonating with the market? The other question I have is whether its practitioners have really left their RUP days behind them. Do they really believe their new-found values? However, given the choice between SAFe and DAD, I would clearly chose DAD, especially if your coaches really understand Agile and the values it represents.

LeSS is more two guys with a book at the moment (3 books, actually). Their approach is sound, but there is no institutionalised marketing. While the Scrum Alliance currently lists 57 Certified Scrum Coaches, the "Black Belt" of the Scrum world, this is not strictly speaking a LeSS certification, nor are any LeSS courses or certifications being offered. On the other hand, that's about where Scrum was 10 years ago. People reading books, and deciding to change. For my money, it's time to start reading Larman and Vodde.

P.S. This article represents a first pass at comparing the frameworks. Does it sound right to you? I would love to hear from practitioners of these approaches to share the experiences and/or improve this article!

Invitation: Open House / The Agile Company / Vernissage

Scrum. Community. Improvement.
Agile. No Hype. No Frameworks.
Training Room. Open House.

Apéro. Vernissage.
The Agile Company.
Join us!

Update: The evening session is now an official Jazoon Side Event! So we have added a Coaching Clinic in the evening.

Want some help with specific challenges you have encountered on your way to a more Agile way of working? Come Coaches Clinic where you can speak one-on-one with an experienced Agile Coach. We can help you across issues like technical practices, organizational change, Scrum, Kanban, Agile Coaching as a career and many other topics.

Invitation

I am proud to start a new series of events on "The Agile Company"... and celebrate my new location!

The Agile Company is a series of events that look at the challenges and rewards of taking Scrum beyond software and becoming and staying an agile company. And a chance to meet and share experiences.

Each month an experienced speaker will present his or her experiences taking Agile beyond software. Then we'll do an interactive session on helping some part of your company become more Agile. And those who want to stay around for an after work Apéro are more than welcome to do so.

Talk: Going the next step? Agile Values and Principles Applied to Hardware

Speaker: Urs Boehm, Head of the Competence Centre Project Management at Noser Engineering

Abstract: Hardware and software development have different requirements for efficient development processes. Properly used, the agile methods can be used successfully here. The speaker shows the differences and shows how agile methods and processes can be used as efficiently as possible for the hardware and hardware-related software development.

Date/Time: October 23, 2013. 14:00 to 20:00, including Apéro and Coaching Clinic.

Location: Training Room "Zurich-West" Hardturmstr. 181, 8005 Zürich (more info)

Afternoon Program:

14.00 Welcome & Coffee
14.30 Urs Boehm - Going the next step? Agile Values and Principles applied to hardware
15:15 Break
15:30 Peter Stevens - Workshop on Applying Agile Values in your company
16:30 Break

No charge, but registration required. Sign up now!

Evening Program / Jazoon Side Event:

17:00 Vernissage with Zurich Artist, Renate Grob
17:30 Apéro starts
18:00 Coaches Clinic with Steve Holyer and Peter Stevens
19:45 Last coaches session at Coaches Clinic
21:00 Apéro moves someplace near by…?

No charge, but registration requested. Sign up now!

Next Month: Is Agile Forever? The Rise and Temptation of an Agile Organization. (more info)

Call for Participation

Do you have a relevant story? Would you like to share it at a future event? Drop me line!

Hotel Discounts for Course Participants

It is always an honor when somebody comes from out of town to attend one of my courses. Thanks to the location of my new training room in Zürich-West, that is even easier, because lodging, public transport and parking are all closeby.

I am pleased to announce that 25 Hours Hotels have agreed that course participants can "receive a 15% discount on the best available rate of the day."

All you need to do is ask me for the booking code.