features
A Methodology to Support Software-Release Decisions
Software is everywhere, and developing it has become a major worldwide industry. We find software embedded, for example, in watches, coffee makers, cars, televisions, airplanes, telephones, reservation systems, and medical equipment. Software not only pervades a multitude of products, but also is an important corporate asset, and demand is increasing. Yet software projects are characterized by schedule and budget overruns and the delivery of unreliable and difficult-to-maintain software products.
Despite an exponential increase in the demand for and dependence on software, many software manufacturers exhibit unpredictable behavior. It is sometimes difficult to determine, for example, a software product’s release date, its features, the associated development costs, or the resulting product quality.
Uncertainty in the release date causes difficulties in planning product promotions, customer training, and maintenance support. Resource utilization across projects can become inefficient and difficult to manage when projects do not meet schedules. Customers have difficulties planning for the introduction of new software into their organizations when a scheduled release date is missed.
The exponential growth of the number of software products and releases suggests that end users will be exposed to more defects if the software industry is not able to reduce defect potentials and increase removal efficiencies at a similar rate. Competitive pressures related to feature content, time to market, and product quality will make product-release timing decisions both more important and more complex.
A Strategic Software-Release Methodology
Software-release decisions often have strategic implications because of the high costs of reversing the decision. Prospective losses also may arise long after the release decision has been made; for example, in cases where liability leads to lawsuits. Existing decision models for product release often do not account for market strategy. Ideally, release-decision criteria should align with corporate strategy.
A software-release decision can be seen from different perspectives:
-
Maximizing behavior. A software-release decision is a tradeoff between early release to reap the benefits of an earlier market introduction and deferred release to enhance functionality or improve quality. If a software product is released too early, the software manufacturer incurs the post-release costs of later fixing failures. If a software product is released too late, the additional development cost and the opportunity cost of missing a market opportunity could be substantial. These two alternatives must be compared to determine which alternative maximizes economic value.
-
Optimizing behavior. A release decision is affected by the difficulty of verifying the correct implementation of functional and non-functional requirements. How much testing is needed? Software manufacturers must find the optimal level of information because information has its price in cost and time. In practice, cost and time will constrain the ability to retrieve complete and reliable information, so this search for information should be considered as an economic activity. This leaves the software manufacturer with the problem of finding the optimal level of information where marginal value equals marginal costs and marginal yield is zero. This optimal level is difficult, if not impossible, to find.
-
Satisficing behavior. Decision-making in the real world is often unstructured and normally involves various stakeholders who may have reasons to release a system or software product because of political or business pressures even though they know that it still contains defects. A study of spacecraft accidents, for example, reveals that, although system and software engineering were inadequate during development, management and organizational factors—including the diffusion of responsibility and authority, limited communication channels, and poor information flows—played a significant role [Leveson 04].
-
Decision Implementation. A decision is considered successful if there is congruence between the expected outcome and the actual outcome. This definition sets requirements for decision implementation. In practice, there are many obstacles to the successful implementation of almost any decision, including
-
-
the reduced importance of a decision once it is made and implemented
-
the control of the outcome of a decision by stakeholders not involved in making it
-
the development of new situations and problems that command the attention of the decision-makers once the choice has been implemented
-
Improving Strategic Software-Release Decisions
To investigate how to improve strategic software-release decision making, the SEI supported research into designing a release-decision methodology. This research reviewed the four perspectives detailed above from both a theoretical and an empirical point of view by studying practical examples. The results helped to frame a proposed release-decision methodology to address software-release decisions from different perspectives. (See the appendix.) The methodology, which consists of a defined set of practices, combines insights from economics, software management, and social psychology.
Studies in three different organizations validated the methodology. One participating organization is a leading global financial services company that provides financial services and products to retail and business markets. Services include insurance, pensions, occupational health and safety, asset management, investments, leasing, real estate, venture capital, and mortgage finance.
The research examined a project in this organization’s IT department, which develops custom systems for internal and external use. The initial estimate for the schedule was 10 months and for the pre-release cash outflows, €15M. Budgets were reserved for the technical infrastructure and post-release cash outflows for maintenance and exploitation. During the first months, the project encountered several setbacks: technical problems surfaced and the development budget turned out to be optimistic. Progress control was lacking, mainly through the absence of clearly defined milestones or quality gates. These problems increased, and in November 2001, the project was redefined. Both senior management and the marketing department exerted pressure on the product-development team to release the product as soon as possible. The team was, however, faced with an unstable product under test and had to use a veto several times to postpone a scheduled release date. When the product was released, uncertainty was high because many known problems were not resolved (although not considered critical), and the organization judged that continued testing would reveal more defects, including potentially critical ones, that could severely hamper the correct functioning and stability of the product.
After the product release, a special task force assumed responsibility for corrective-maintenance activities. This team needed more than a year to resolve the known and newly detected defects. Despite the original requirement to develop a maintainable product, the organization decided in 2004 to start a pre-study toward a new product to replace this product because corrective maintenance and functional enhancements proved difficult and costly. In other words, the early release of the product saved the organization additional testing costs, but the post-release maintenance cost turned out to be significantly higher than expected. A retrospective review of this project using the release-decision methodology enabled this organization to assess the project from a release decision point of view.
Figure 1 illustrates how the organization scored on the identified practices in the methodology. Lack of a product-development strategy (release definition) and lack of information as input to the decision-making process (release information) led to a poorly structured release-decision process without consensus among the stakeholders involved (release decision). Sufficient financial resources saved the organization in the short term by making it possible to patch the released software.
Figure 1: Radar Presentation of Case Study Results
These results from a practical setting indicate that this software-release decision methodology supports understanding, analysis, assessment, and improvement of the capability of software manufacturers in this problematic area. Research in software-manufacturing environments is under way to study the effects of applying the methodology at the start of projects to proactively aim for release-decision success.
For further information, see the full study or contact—
Hans Sassenburg
Email
hanss@sei.cmu.edu
Jay Douglass
Email
jcd@sei.cmu.edu
Reference
[Leveson 04]
Leveson, N.G. “The Role of Software in Spacecraft Accidents.”
AIAA Journal of Spacecraft and Rockets, 41, 4, 2004. Appendix:
Release-Decision Methodology
About the Author
Hans Sassenburg works as a visiting scientist for the Software Engineering Institute in Europe. He is a part of the Software Engineering Process Management group. In 2002 he started doctoral-level research studying the area of software release decisions. This work was finished in January 2006.
|
