Sunday, February 21, 2010

In Praise of Middle Management

On a recent post on on Scrum's (alleged) shortcomings,  Bob Martin (aka Uncle Bob) wrote:
Scrum carries an anti-management undercurrent that is counter-productive. Scrum over-emphasizes the role of the team as self-managing. Self-organizing and self-managing teams are a good thing. But there is a limit to how much a team can self-X. Teams still need to be managed by someone who is responsible to the business. Scrum does not describe this with enough balance.
I agree with Bob: Self-organizing teams can focus too much on themselves. This can manifest itself as an uncooperative relationship with other teams and/or as an antagonistic relationship with management.

In a theoretically perfect Scrum organization, there might not be much of a middle management layer, just Product Owners and ScrumMasters guiding their interdisciplinary teams on a quest of building better products and services while continuously improving their ability to do so.

The current state of the organization is much more likely to be a hierarchy organized on functional boundaries. The Scrum organization is potentially a factor 10 more effective that the current structure, but reorganization is dramatic, scary, painful and not without risk.

Scrum enables change. Scrum demands change. The essence of Scrum is change. Very quickly, the Scrum Team (or Teams) will come into conflict with the structures the company has in place. These structures are not anybody's fault. At the time they were created, they seemed like the best thing to do.

Change demands leadership. Who provides that leadership? Middle management.

Middle management (and probably one or two middle managers) is the unsung hero of a successful Scrum transition. Middle management is the source of lasting change in an organization. Initiatives from top management often fade as soon as the relevant manager changes roles.  Operational staff is too powerless to instigate change.

Middle management is caught between the rocks and the hard place. The "rocks" are the demands from above and the "hard place" is the reality of accomplishing those goals.  Middle management has influence, especially the 2nd level and up. When middle management recognizes that the methods and structures currently in place are not competitive, they look to something better.  That something is Scrum.

Middle management does not have the power to command change in the organization, so it has to negotiate and build consensus about how to move forward. Eventually, when that consensus is achieved, someone at the top will bless it, and there will be a new org-chart.

Scrum means tremendous change in the way the organization works and eventually, the way it is structured, too. So the middle managers driving the change also have the most personal risk. Ironic isn't it? Why do they do it? Because the company needs to be better than it is. (And they needn't worry - there will always be room for managers who can make their company better than it is!)

Once I realized that middle management is the true source of leadership and why they do it, my respect for the importance of middle managers grew tremendously. I have banned Dilbert and the Pointy Haired Boss from my presentations. And I encourage my coaching customers to realize the importance of their mission and the importance of their role in accomplishing it.

6 comments:

Daniel Marbach said...

Hy Peter,
I definitely agree about the middle management part. But I would say not only! In the current project I'm working also the upper management stands behind scrum and our team. Because the old software grew in a non agile, non continuous and non test driven environment the product was unstable and not extensible. With scrum and XP pratices like continuous integration also the upper management sees the benefit for long term projects and totally supports us. This means the middle management has the backup in case other departments of the company develop jealousy (Oh yes that happens!) and begins to act politically against our team (Oh yes that happens too!).

Daniel

Peter said...

Hi Daniel,

You live in Agile Heaven!

I agree completely: life is much easier with top management on your side. I would consider it the goal of any transition project to get top management on board, if that is not already the case.

Before you get there, there are some dysfunctions to watch for: 1) waiting for blessings from top management before doing anything, 2) expecting top management to come and solve the problems, 3) using lack of blessing (or even an order) from top management as an excuse to do nothing.

Cheers,
Peter

PM Hut said...

Part of the problem of scrutinizing SCRUM is that it is an offshoot of Agile, which has it own limitations. #2 in the linked article is: "Fit with organizational culture". Agile (and Scrum) has to smoothly fit in the organization, the team has to be a qualified team (a team of stars). If you don't have the right team, you just can't do Agile, and consequently Scrum. That is one of the main problems with this methodology.

Peter said...

Hi PM Hut,

Interesting arguments, but they miss the point of Scrum.

Agile is a very broad concept. Scrum is a very precise set of practices and principles.

Scrum does not claim to fit into any organization. Scrum is a framework for changing the organization for the better.

Is it for your organization? Depends. Do you want change? If the answer is no, then don't even think about Scrum!

Does Scrum need of team of stars? No. Scrum is about unlocking the potential of the people you have. If they have no potential, Scrum will also make this clear. If your people have no potential, no methodology will really help you. Fortunately, this is a very rare situation.

My experience with software development large organizations suggests that most of those organizations could be a factor of 10 more productive than they are.

Roughly half of the improvement comes from getting the implementation team to work more effectively (sometimes even taking the team star off the team is a key to achieving this improvement).

The other half comes from improving the effectiveness of their management.

If your company has a lot Dilbert cartoons hanging from people's walls, then you probably need change and Scrum will help you fix your problems. But only if you want to fix them.

Ishwara Bhat said...

I am excited to see this blog post. I saw some excesses of SCRUM trained people at work place and hesitatingly wrote a post in my blog. Some of them even conflict with agile manifesto.
In practical world, we do not get all stars. Also the business realities need to be accounted for.

May ever practitioner read your post. Good for the world!

http://midmanager.com/agile-scrum-and-communism-must-follow-survival-guide-for-your-organization/

Peter said...

Hi Ishwara Bhat,

Thanks for commenting and your for blog post. The situation you describe is not Scrum or Agile, but it does happen from time to time.

I would not characterize either Scrum or Agile as communist, because they are about creating value for customers and stakeholders.

One of the difficult concepts to understand is how the team is both part of the organization and protected from undue interference from elsewhere in the organization (or even from the product owner).

Another concept which can be difficult to accept is that the people doing the work can a) be motivated without coercion and b) understand the work better than their managers. (This may not have been true when the assembly line was invented, but it is typically true for products created today by knowledge work).

Teams need to learn that even though they have the necessary authority to get their jobs done, they are still part of the organization and must cooperate with the rest of the organization.