ValueShepherd.com: Thoughts on R&D Management

Commentary

 Home

Commentary
Retrospectives

About

Contact

Retrospectives

"If I had one wish"

Daniel Kahneman pioneered the field of behavioral finance, and recently received the Nobel prize in economics for that work. One of his greatest discoveries is that we all tend to make irrational business decisions.

So, here's a man who has spent the last few decades studying how irrational we are, and what one thing does he think we should do to improve our decision making? [1]

Use formal decision analysis? No-- he says it requires more discipline than nearly anyone can sustain.

Improve our teamwork? Probably not--he notes that groups reach more extreme conclusions than do individuals.

Do better risk analysis? No again--managers don't want to admit that they aren't in control.

He recommends nothing as sophisticated as any of these ideas, and instead says "if I had one wish, it is to see organizations dedicating some effort to study their own decision processes and their own mistakes, and to keep track so as to learn from those mistakes."

Daniel Kahneman is calling for project retrospectives.

It's always nice to agree with a Nobel Prize winner!

The Fuzzy Front End of Process Development

I consider retrospectives to be essential to ongoing improvement in any organization of any size, even for individuals. In my opinion, they are a very effective way to manage the fuzzy front end of process improvement.

Fuzzy front end? The term "fuzzy front end" usually refers to the early stage of product development, the time when a team sifts through many ideas, makes major investment decisions, and defines the focus for a project. One must carefully manage this key phase to increase a product's chance of success [2].

As figure 1 illustrates, process development also has a fuzzy front end.

Just like product development, process development starts with sifting through many ideas and searching for something to focus on. As the diagram shows, the fuzzy front end of process development begins as a project ends, and it provides process changes to the fuzzy front end of product development. Process development feeds work to and receives feedback from product development.

Project retrospectives are an effective tool for managing the fuzzy front end off process development. They are a way for companies to "study their own decision processes and their own mistakes", as Kahneman says, and to quickly identify attractive process improvements as well.

Conversely, any organization that doesn't hold retrospectives probably isn't managing the fuzzy front end of process development and may have no ongoing process improvement at all.

Retrospective Agendas

I am surprised that more companies don't use retrospectives. A recent study reported that "only one out of five R&D projects is reviewed after termination, thereby losing important learnings". [3] Getting started on retrospectives is easy. A basic retrospective, one meant to stimulate a team to generate ideas, follows a simple agenda:

The Basic Retrospective Meeting

Describe the project's objectives
Describe the project's outcome and history
Evaluate the difference between outcome and objectives
Brainstorm what to do more of, less of, or differently next time
Clarify and add to the ideas
Define categories of ideas
Prioritize and select categories to work on

This basic retrospective works fine for a team of people who work well together, know their objectives, and can implement their own process improvements. A manager or team leader with basic facilitation skills can run a basic retrospective very effectively, and they usually take between four and eight hours to complete.

More advanced retrospectives build on this basic agenda. For example, a team that needs to connect its project to its organization's larger goals and then propose process improvements to senior management could use this agenda:

A Retrospective for an Embedded Project

Pre-meeting:
• Collect feedback from internal customers
• Review and summarize original objectives
At the meeting:
• Basic Retrospective
• Create process improvement business cases
Post-meeting:
• Present business cases to senior management

This kind of retrospective requires more preparation. An outside facilitator is useful to enable managers to participate fully in the review. These more complicated retrospectives take one to two days to complete.

Retrospectives are great opportunities to do team-building. For a team that really needs some team building, a facilitator is essential, and an agenda would be:

A Team-Building Retrospective

Pre-meeting:
• Facilitator meets with key opinion leaders to identify problem areas, find common ground, and define ground rules
• Facilitator, opinion leaders, and management-sponsor define ground rules for meeting
At the meeting:
• Review ground rules for conduct
• Summarize team successes (if there are any, this sure helps)
• Basic Retrospective
• Use carefully-chosen sub-teams and exercises to address team schisms
• Mix in non-project activities during off-hours
• Team presents findings to sponsoring managers
• End with an exercise that reinforces improved working relationships

Norman Kerth's book Project Retrospectives: A Handbook for Team Review [4] provides an excellent menu of retrospective meeting practices and is an especially good guide for team-building retrospectives. He recommends spending as long as three days on a retrospective.

There are many other sources of exercises to use in retrospectives. Three that I've used are Mining Group Gold [5] and de Bono's Thinking Course [6].

Pitfalls

Of course things can go wrong before, during, or after a retrospective. Here are some common problems and solutions that I've used:

Dysfunctional teams. There's a minimum level of mutual respect that's needed for a retrospective to work. Fortunately, that minimum level is not too rare, and there are some simple techniques to encourage it.
• Clearly explain roles and the agenda at the beginning of the meeting and stick to them. Provide a lot of structure so that much of the team's attention is on the process rather than on each other.
• Record every idea and remark on a flip chart. This has two beneficial effects. First, it reassures people that their ideas will not be forgotten, which reduces competition in the room. Second, it discourages outrageous and personal comments--those who offer them soon realize that when the time comes to clarify the ideas on the wall they will have to explain their inappropriate remarks. Many times I've had people sheepishly request that their comments be deleted. Natural consequences are a great facilitating tool!
  
The desire for comfort. We all tend to suggest actions that will make our jobs easier or more fun. Often, however, those ideas will not make our companies more profitable. Counteract this tendency by spending some time during the categorizing and selecting step of the basic retrospective agenda to remind the group of the business objectives of the retrospective, and to introduce them to the investor's perspective.
Lack of trust in managers. Managers at all levels should be free to participate or drop in on a retrospective. If a team is unable to contribute enough useful ideas with a manager present, then there are three options:
• Assess the trustworthiness of the suspect managers. If the problem is the team's perception, coach the managers (separately and in advance) on how to participate in the retrospective, and then proceed. Use the retrospective to show the team that they can trust the managers, at least during a retrospective.
• Find and address the root cause of the lack of trust, whether due to the managers or the team. This is an ambitious goal that probably requires an organization development consultant to achieve.
• Organize the retrospective so that the managers participate only at the end as the customers of the team's final recommendations. Then require the team to meet a high standard of conduct and refrain from inappropriate remarks about the managers during the retrospective--disrespect for a customer is never appropriate, and gossip derails retrospectives.
Group-think. As Daniel Kahneman observes, groups tend to reach more extreme conclusions than individuals. This is a greater problem for close-knit teams. To counteract this tendency, include interested outsiders in the retrospective, such as suppliers, partners, internal customers, and internal service providers from IT, Finance, or HR. Coach them in advance on their role as sanity-checkers, and then encourage them participate fully in the retrospective.
Unclear authority. Make clear to the team before the meeting what influence they should expect to have on decisions to fund process improvements. If the team is its own customer, great. On the other hand, if senior managers will review the team's recommendations and possibly fund no improvements, then the team must know that up front. Failing to clarify authority will create disdain for retrospectives. On the positive side, teams that know they must prepare compelling investment proposals get energized by the challenge.

The Power of Retrospectives

Retrospectives have an intuitive appeal, but to create value they must solve business problems and create economic profit. The retrospectives that I've facilitated and taken part in have identified practical solutions to problems such as:

disconnects between market requirements and project deliverables
slow product development
slow product launches
poor product quality
unexpectedly high manufacturing or service delivery costs
serious disconnects with partners and customers

Of course, solving many of these problems requires disciplined follow-through in senior management (the subject of another article); however, one beauty of retrospectives is that there is value in identifying the top opportunities for process improvement and in the team building that occurs en route, even if senior management never follows through. I've often been impressed by the initiative that people take, after a retrospective, to implement valuable changes themselves. Examples from my experience are:

a team that created a development process by organizing retrospective ideas into simple checklists.
software engineers who implemented peer-programming practices during the end-game of development.
a team that documented a process for working with third-parties and submitted it to a corporate quality office.
slow-but-steady improvement in support for resellers.
a shift in focus, throughout a team, from the details of engineering to defining customer requirements.

Obviously, the best outcome is for senior managers to fully support and follow through on retrospectives--they provide a rich source of investment ideas. In addition, though, retrospectives create value by encouraging initiative in project teams and supplying those motivated people with ideas for valuable grass-roots improvement.

The Distinguishing Practice

There are countless philosophical quotes that support using retrospectives, from sayings about "not repeating the past" to mystical claims like "everything you need to be happy is within you". Here, instead, is a thought-provoking, very practical story.

Jim Loehr, author of Stress for Success [7], is a sports psychologist. Several years ago he studied top tennis players to determine what separates the very top players (those who stay close to the #1 ranking) from the rest of the field.

What set the best apart was not conditioning, stroke mechanics, strategy, coaching, or any other variable one might expect. Instead, it was what the top players thought about in between points.

In product development and project management there is a lot of emphasis, really all possible emphasis, on what to do during a project. I have to admit it's a leap from Loehr's finding to a conclusion about the performance of organizations, but I based on my experience I think that what sets the best product development organizations apart is what they think about in between projects.

Good project management is essential and a worthy investment; however, to separate your product development capability from the crowd in your industry, look elsewhere--invest in project retrospectives, and use them to drive improvements in your project management.

References

1. “Daniel Kahneman: The Thought Leader Interview”, Strategy+Business, Winter 2003, (accessed 12/21/2003), http://www.strategy-business.com/press/article/03409?pg=0 (registration required).

2. P.G. Smith and D. G. Reinertsen, Developing Products in Half the Time 2^nd Edition,  John Wiley & Sons, New York, 1998.

3. M. von Zedtwitz, “Post Project Reviews in R&D”, Research•Technology Management, September-October 2003.

4. N. L. Kerth, Project Retrospectives: A Handbook for Team Reviews, Dorset House, New York, 2001.

5. T. A. Kayser, Mining Group Gold: How to Cash in on the Collaborative Brain Power of a Group, Irwin, Chicago, 1995.

6. E. de Bono, de Bono’s Thinking Course, Facts on File, 1985.

7. J. Loehr, Stress for Success, Three Rivers Press, 1998.

Copyright 2003, Peter Bradshaw. All rights reserved.