Commentary
Commentary
Situational
Process
Situational Process
|
Summary |
Process is not religion.
It isn't even radical politics.
But, even some of the best writing on process sometimes treats it as if it were. Here are some examples:
- "Is Design Dead? Seeking Salvation with Extreme Programming." This is a sensible article in Software Development magazine about the value of design for teams using extreme programming. To be fair to the author, it looks like the magazine editors added the reference to salvation just on their cover, but why is it so tempting to use religious terms when we discuss process? And did people really think that design skills were obsolete because of extreme programming? [1]
- "The Agile Manifesto." This is an interesting proclamation in Software Development magazine by a group of people who refer to themselves as "seventeen anarchists". The ideas are worthwhile, but why the anarchist spin? Many of their ideas are not new, or even very radical. [2]
Both of the above articles offer valuable ideas, including the notion that practices must differ on different projects. Unfortunately, the titles and, to a lesser degree, the tone of these articles and others like them encourage a rigid, I'm-putting-a-stake-in-the-ground kind of attitude. The authors of the Agile Manifesto noted that they were quickly asked if the manifesto would become a "Unified Lightweight Methodology", as if by publishing their decree they could end the agile process dialogue and harden one set of rules for everyone to follow. How ironic that an article on agile process would encourage such rigid thinking. The authors, of course, corrected that misunderstanding.
Opinion
There is only one invariant in process: what works is entirely determined by the situation. Even principles as sensible as those in the above two papers have counter-examples. "The Opposite of a Profound Truth is Also True" is the title of one chapter of the book Management of the Absurd, an entertaining exploration of paradoxes in management [3]. What this means is that while there are certainly some worst practices to avoid in all situations, there are no best practices. Not only that, but even some of the most common objectives in system development, such as user satisfaction, may not always apply [4].
Don't Adhere, Think!
An intelligent product developer does not adhere to a particular practice or methodology, but instead thoughtfully selects the practices and methods that work in a particular situation, and then applies those practices with discipline. By "thoughtful", I mean taking care to consider that even the most hallowed beliefs in product development may not apply to your project. A particularly thoughtful developer might even encourage contradictory practices within a single project, depending on the situation at the moment. Matching practices to the situation is "situational process".
Situational process is consistent with the idea of situational management, a model that encourages a manager to tailor how he or she works with an employee depending on the employee's ability to work independently. Similarly, a management team, and subsequently a project team, should take into account the situation when staffing and organizing a project.
Note that the idea of situational process does not imply that project teams should invent their practices. There is a wealth of practice to draw on, and scant need for inventing new ones. Project teams do need to thoughtfully select practices to use, though.
Practice
But what factors should a manager or team take into account when selecting practices? In situational management, there are two factors: task maturity, or the ability of an employee to perform tasks, and relationship maturity, or the ability of an employee to work comfortably as well as effectively with other people.
There are many factors that affect the fit of a project team with its process. People, technology, customer behavior, and team size are just a few. However, it would be nice to have a two-factor model for analyzing how well various practices are likely to work for a given project. Such a model would enable picture-drawing and simpler decision-making. I think there is such a model.
Two Key Factors for Assessing Practices: Cost of Change and Probability of Change
The two key factors are the cost of change and the probability of change. The cost of change factor appears frequently in the literature about agile methods[1,2,4], the usual point being that agility both requires a lower cost of change and reduces the cost of change. A low cost of change is key to making a large number of mistakes as fast as possible as a team builds a product incrementally.
The probability of change is not cited as often but is just as interesting. Not all projects can reduce their cost of change and be agile. In those cases, reducing the probability of change is the right course of action.
Here are some definitions of the two factors, based of course on the investor's point of view:
The cost of change is the impact on a product's ROI of introducing a change. A change is any interruption that causes a team, or a part of a team, to fall out of alignment, i.e. it causes the team to stop thinking together. The cost of change is then the investment required to bring the team back into alignment. Note that a team can do a superb job of implementing a change but still incur a high cost of change. Also, by using the investor's perspective in analyzing the cost of change a team will take into account the cost to all stakeholders, and thus the impact on the total value created by the project.
The probability of change is the likelihood that changes will occur. Changes may originate externally, for example due to a volatile market, or internally, due to mistakes or intentionally deferring decisions.
Some idea of what constitutes a high cost of change or probability of change will enable us to apply these two factors to selecting practices. For our purposes the cost of change for a project is low if changes only rarely affect the investment performance of the project (e.g. less than 5% of the time). In other words, if the team can make 95% of needed changes, including the implementation of those changes for all stakeholders, without affecting the investment performance of the project, then the cost of change is low. Any team that doesn't fit this mold has a high cost of change.
The probability of change is low if the team is able to establish task lists as part of project planning and then execute those tasks without change.
The Two Factors Interact Strongly
Many projects reduce the probability of change by increasing the cost of change. One such approach is to write contractual specifications at the beginning of a project and implement a detailed change control process. Both of these actions increase the cost of change and reduce the probability of change. This approach can increase risk as well because the probability of getting the contractual specifications wrong at the beginning of a project is often high. This is risk due to making decisions in the absence of information.
Conversely, sometimes a project team can increase the probability of change to reduce the cost of change. This works when deferring decisions enables better decision making in the future. The team accepts that it will have to make changes, but knows that it will do a better job of implementing those changes in the future when more information is available. This is a foundation of agile methodologies. Note that this approach may also increase risk. The risk in this case is due to allowing team members to make assumptions until they can collect the information required to make good decisions.
When Agility is Clumsy, and Vice-Versa
Recently, some software methodologists have embraced the agile approach, and sometimes branded more traditional approaches as too rigid, even obsolete. But the agile approach does not always work. Not all situations permit reducing the cost of change. Some situations demand carefully controlling change because the cost of change is unavoidably high. As it turns out, good product developers have employed both agile and heavy processes for decades. The Lockheed Skunkworks is a good example of a team that employed both heavy and agile practices with great results.
The model below uses the two factors of cost of change and probability of change to organize a selection of software engineering practices. The menu indicates which practices are appropriate for a given situation.
The model positions some common software development practices according to whether they reduce the probability of change, reduce the cost of change, or both. The Y axis indicates the probability of change, and the X-axis the cost of change. The model guides a team in combining practices according to their situation, rather than prescribing a particular process.
In the absence of appropriate structure, teams tend to wander toward the upper right quadrant of the model, i.e. they create both a high cost of change and a high probability of change for themselves - not a good place to be. From that upper right quadrant, teams can invest in three kinds of practices to improve their performance: those that reduce the cost of change, i.e. build agility, those that reduce the probability of change, i.e. build commitment, and those that improve the team's judgement, i.e. the efficiency of their interactions and decision making.
Invest in Appropriate Practices
By taking the investor's point of view, a team will be conscious that adopting a practice is an investment that has its own benefit, cost, and risk. The menu helps a team decide which practices are likely to be good investments.
For example, a small team that is building a website for internal use is likely to have a low cost of change, frequent builds, and some design latitude. That makes the practices in the upper-left of the menu a good fit for them. Conversely, a team building a military satellite has an unavoidably high cost of change due to tooling, fan-out to many disciplines in a large organization, and a very conservative, careful customer with a high-cost change control process. Such a team should start with the practices in the lower-right of the menu.
Either team could consider the practices in the lower left corner to improve its team performance. Also, both teams should consider using practices in the opposite quadrant to solve specific problems. The web site team could use contracts, use cases, and customer testing to structure its interaction with a volatile but visionary sponsor. Even agile teams, after all, can get fed up with change. The satellite team on the other hand might be able to define a component sub-team that, given a rigid interface for its component, has the freedom to be agile within the confines of that interface.
Next, teams may benefit from viewing the practices in the opposite quadrant as a menu for skill development rather than process. For example, extreme programming requires its practitioners to continuously make design judgements. That's a high-level of design sophistication. Agile developers need to apply traditional skills frequently, quickly, and thoughfully - that's what agility is all about. Engineers in highly-structured projects can benefit from practices analogous to pair programming. Traditional developers need to be on the lookout for opportunities to move quickly within the constraints of their project.
A team can use the menu to position and assess a new practice and its fit with the team. For example, a team considering Scrum's [5] idea of 30-day sprints should ask how that practice will affect change, and then assess its value to the project. A 30-day sprint is a technique for reducing the probability of change. It creates a 30-day mini-project with an up-front specification. During that 30 days, though, the team can employ agile practices to meet that specification.
When using a practices menu, a team must take into account the interactions among practices as well as the fit of each practice with the team's situation. Kent Beck gives some very good examples of practice interaction in Extreme Programming Explained [6]. A team needs to get aligned and stay aligned, and certain combinations of practices encourage alignment, while others permit anarchy. For example, an agile team can stay on track by combining story boarding (the vision) with pair programming (to check understanding and keep the code clean) and frequent builds (to check alignment on interfaces). All three practices encourage face-to-face confirmation that the team members are thinking together. However, leaving out any of the three is likely to create confusion regarding either requirements or implementation.
The menu approach to practices encourages intelligent, situational management of product development. It employs an investor's perspective in assessing the utility of specific practices, and helps teams tailor their process.
Many methodologies assert that their particular combination of practices is optimum, and that any change to their special combination will fail. Don't believe them. Such claims introduce a thoughtless religion to product development, and lead to useless debate as well. Organize the union of all practices discussed in the leading books and journals into a menu, categorize those practices according to what they do for you, and then pick what will work for your project from that menu. The change-based model above is a good way to start that analysis.
Philosophy
Is the menu approach to process development philosophically sound? Definitely.
Claims that a particular process is "the way" are no more valid than similar claims about centrally-planned economies (which, in case you haven't noticed, failed late in the twentieth century). The world, even very small parts of the world, are too dynamic for anyone to pre-specify how to run projects in all situations, or even in particular types of situations.
Summary
- Consider your situation before you consider process. Assume that your situation is unique - the burden of proof is on those who claim it isn't.
- Take into account all stakeholders when assessing a project's fundamental cost and probability of change.
- Construct a practice menu based on the cost of change and the probability of change. Other factors are also useful in subsequent steps, but these two are the foundation for analysis.
- Select practices so they fit the situation and assess their interactions to ensure they will encourage team alignment.
- Take the investor's perspective in assessing your investments in process.
References
1. M. Fowler, “Is Design Dead”, Software Development, vol. 9, no. 4, April 2001, pp. 42-46.
2. M. Fowler and J. Highsmith, “The Agile Manifesto”, Software Development, vol. 9, no. 8, August 2001, p. 28-32.
3. R. Farson, Management of the Absurd, Simon & Schuster, New York, 1996.
4. D. P. Truex, R. Baskerville, and H. Klein, “Growing Systems in Emergent Organizations”, Communications of the ACM, vol. 42, no. 8, August 1999.
5. ControlChaos.com, (10 September 2002) < http://www.controlchaos.com/>
6. K. Beck, Extreme Programming Explained, Addison Wesley, Reading, 2000.
Copyright 2002, Peter Bradshaw. All rights reserved.
The right to download, store, and/or output any material on this Web page is granted for viewing use only. Material may not be reproduced in any form without the express written permission of Peter L. Bradshaw. Reproduction or editing by any means, mechanical or electronic, in whole or in part, without the express written permission of Peter L. Bradshaw is strictly prohibited.