A Framework for Software Product Line Practice, Version 5.0

Tool Support

Software development organizations use tools to support many of the activities necessary to transform a set of customer needs into a useful product. A multitude of computer-aided software engineering (CASE) tools are available to help automate the analysis, design, implementation, and maintenance of software-intensive products. The challenge is to choose and use tools wisely to support the business goals of the organization and the technical needs of the product developers.

Typically, many tools are used over the life of the project. The degree to which they interoperate and support the development process can have a large effect on the productivity of the development team and the quality of the resultant products [Bruckhaus 1996a, Low 1999a]. Standards have emerged to provide guidance in the evaluation, selection, and adoption of CASE tools [IEEE 1998a, IEEE 1996a, ISO 1995a], and several authors have described approaches to the tool integration problem and the results of tool adoption studies [Brown 1994a, Brown 1994b, Bruckhaus 1996a, Budgen 2003a, Favre 2003a, Jansen 2004a, Low 1999a, Maccari 2000a, Powell 1996a, Vollman 1994a].

Aspects Peculiar to Product Lines

If you're building a software product line, a variety of tools will likely be used, since there is no single tool that addresses the needs of all product line practices. While many of the practices supported by tools are applicable to product lines, the product line context brings some of its own particular needs and risks of its own.

This practice area does not address the related issue of choosing and using technologies (such as distributed object technology) that may be new to a product line organization and for which tool support may be required. The focus here is on applying familiar tools to practices in a product line context.

The critical aspect of this practice area is support for concurrently creating, maintaining, and using multiple versions of product line artifacts–both core assets and products. That support requires a development environment that facilitates the coordination of core asset development and product development teams and processes and the sharing of core assets among teams.

Tool support for a product line is therefore more than the sum of the capabilities of individual tools that support specific software engineering activities. The interoperability of a chosen set of tools is critical for the automated production of products in a product line from core assets. An integrated product line support environment will

support the product line development process and the organizational structure that implements it
enforce the conceptual integrity of the product line based on the definition of what it means to be a member of the product line, including any options for tailoring a product to specific needs without violating the defining properties of the product line. That definition comes from the "Scoping" and "Requirements Engineering" practice areas.
represent the common and variant capabilities of products in the product line in all places where they are articulated: requirements, architecture, components, testing, plans, and elsewhere
maintain traceability and dependency links among core assets and products, for product line maintenance and evolution, through a configuration management (CM) capability that covers the entire product line
support an architecture-based development process with multiple views of the product line architecture that permit reasoning about architectural qualities
support the "steady state" operation of the product line by enabling the production of specific products from a specification of their desired capabilities
provide insight into the business and technical decisions for the product line by supporting technical management practices such as scoping and measurement
evolve with the product line to incorporate new technologies and new production strategies and processes as the organization's needs change

Together, these items represent a vision of an integrated product line tool environment. Although we can expect reality to fall somewhat short of this vision, remembering the ideal helps us nudge the real environment in that direction. Jansen and Bosch, for example, examined the issue of tool support for architectural evolution [Jansen 2004a]. They rated five tools against the requirements needed to support architectural design decisions and, not surprisingly, found plenty of variation in the results. Their conclusion was that architectural tool support does not view evolution as an inherent part and distinct dimension of software architecture.

Establishing tool support for a product line involves the following activities:

identification of needs: When choosing tools to support a product line effort, be clear about which specific needs the tools are to address, including the expectations of the various people who will install, use, and maintain them. Don't overlook the essential need for documentation of the product line–for example, describing the business case for the product line, describing the variation points in the product line architecture, and describing how a product engineer creates products from core assets. The future needs of the product line are also a consideration: investigating how new technologies might support the product line, identifying emerging tools in these new technological areas, and communicating future needs to tool vendors.
selection: Base your tool selection on well-defined criteria rather than on market literature or anecdotal evidence of suitability. The specific practices suggested below offer a variety of criteria.
evaluation: Ideally, when assessing a tool for fitness of purpose, conduct a trial use before making a final commitment. A tool may look promising because of its claimed ability to, say, support architecture variability, but a trial use may show that variability to be incompatible with what the architects need to reason about a product's structural properties. Similarly, interoperability and process-support considerations may nullify any gains made through the satisfaction of particular needs.
insertion: Insertion includes adapting tools to the environment in which they will operate, training tool users and maintainers, and adapting or adopting the development or business processes and practices that are necessary for the effective use of the tools.
measurement: The only way to determine a tool's benefits and limitations is to systematically track the effects of its use on an organization's productivity and product quality. The appropriate data should be collected and compared with prior experiences to determine the tool's value in practice.
maintenance: The environment provided by tools needs to evolve with the product line to incorporate new technologies and new production strategies and processes. When evaluating a tool, be sure to consider whether it can be evolved and whether its vendor is open to making evolutionary changes.

Application to Core Asset Development

Tool support for core asset development focuses on the commonality and variability of the core assets from which product line instances will be created. An ideal tool set for core asset development supports the development of both core assets and mechanisms for using them to construct products.

Tools are needed for developing all sorts of core assets, including those for constructing documents, code, test support, and installation–in other words, those that support adopted product line practices. For example

Product line scoping practices are facilitated by tools that capture and represent the common and variant capabilities of products in the product line.
Requirements engineering practices for product lines necessitate tools for describing the requirements of the products in the product line.
Architecture definition practices for product lines require tools for describing the structures of products in the product line in a way that is consistent with the identified commonality and variability for the product line.
Component development practices for product lines require tools for creating and testing components that are consistent with the identified commonality and variability and can be used to construct products.
Production plan practices require tools for creating mechanisms that support the development activities of product development teams.
Product production using model-driven approaches such as domain-specific languages and automatic code generation requires tools for constructing the languages, creating the models and product specifications, and generating code from the models.

Application to Product Development

The exact role of tool support in product development depends on the degree of automation that has been decided on during core asset development and production planning. The development of products may be partially or wholly automated; in the extreme, the products may actually be generated by tools that are created specifically for the product line. In that case, tool support is extensive; the software engineering environment is a production constraint that must be factored into the architecture design and component development for product developers, and it is a significant element of the production plan.

The means of product development are specified in the product line's production plan. Tool support for product development is a particularly important part of that specification because the chosen practices and tools must be compatible. The nature of the production plan can differ substantially depending on the degree to which product development activities are to be automated [Chastek 2002b, Chastek 2002c].

Example Practices

Current tool support for product lines is based on a variety of tools used in the software engineering activities of conventional development efforts and then "stretched" to accommodate product lines [Bass 2000a]. Using these tools successfully in a product line context requires practices that emphasize the tools' fitness of purpose (both individually and collectively), the quality of the software produced using them, their effect on the product line development process, and the business benefits of using them. If an organization's workforce is geographically distributed, tool selection and enforcement must be an early consideration.

The practices discussed below are all based on established practices for CASE tool support and adapted to the product line context.

Identification of needs: In addition to tools that directly support software development, product line tool support is essential for CM, planning, and all types of documentation. All of these activities are good places to begin identifying specific needs.

Selection: Bass and colleagues provide a comprehensive checklist of tool selection criteria for software product lines [Bass 2000a]. Example criteria include tool features, cost, vendor stability, training, interoperability with other tools, and process implications. Powell and colleagues describe an evaluation process that generates selection criteria from a checklist of issues [Powell 1996a], and the longest section of the International Organization for Standardization (ISO) standard 14102 [ISO 1995a] deals with the characteristics by which tools can be selected and evaluated. Other practitioners advocate tying the selection and integration of tools to the development process to be used [Brown 1994a].

Evaluation: Brown advocates an evaluation approach based on the desired quality attributes of the entire tool support environment, not just the individual tools [Brown 1994b], and Garlan, Allen, and Ockerbloom describe integration woes that have particular relevance for product lines [Garlan 1995a]. The evaluation process described by Powell and colleagues [Powell 1996a] and the Institute of Electrical and Electronics Engineers (IEEE) standard 1348-1995 [IEEE 1996a] include guidelines for the pilot use of a tool. Favre and colleagues observed that even when tools are quite close to solutions and based on well-defined concepts, there are still many important barriers to adoption; those authors list a range of issues from usability and customization to evolution and organizational strategy [Favre 2003a].

Automation: Automated derivation of products requires an explicit tool chain that begins with domain concepts and ends with executable code [McGregor 2005b]. There are several paths between these two end points, and the tools are somewhat different along each one. Basically, automated derivation is based on some type of modeling language that is specific enough to have a well-defined semantics yet at a high enough level to allow direct involvement of customers. McGregor lists five main approaches to automatic derivation:

Specification-Based
Intelligent Build
Domain-Specific Languages (DSLs) and Product Generation
Metamodeling
Frame Technology

These approaches are not at the same level of abstraction, and they are not mutually exclusive. Each one requires tool support. Consider two examples that differ in the grain size of the units of derivation:

DSLs provide a relatively fine-grained representation from which products can be developed [Czarnecki 2005a]. Using tools that support domain modeling approaches such as feature modeling or the more general conceptual modeling, a model is created using a vocabulary specific to a domain. Domain experts can then specify products in a language they understand. These modeling tools are augmented by a variety of other tools that provide a means for expressing constraints, code generators that map model elements to specific code fragments, model simulators that allow execution of the model, model verifiers, and model-driven testers. All of these tools are needed for a complete development environment, and many of them can be generated automatically from the domain model.

Batory defines one particular tool chain for product specification that starts with a tool that uses a feature model configuration to specify a product [Batory 2005a]. The model is maintained in a Logic-Truth Maintenance System (LTMS) and uses a propositional satisfiability (SAT) solver to prevent inconsistent specifications. The feature-based specification can be mapped onto a grammar from which various techniques can be used to produce products.
Frame technologies are an easy way to begin automation in a product line because they tend to be used with large code fragments. These techniques generate products and assume that code fragments have been created so that they can be combined in a variety of ways to make many different products [Zhang 2005a]. The tool support for this approach is based on Extensible Markup Language (XML). The overhead is small because most XML-oriented tools can be used with a small specialized engine.

Both domain-specific languages and frame technologies are based on the use of metamodeling techniques. Metamodels provide layers of abstraction that hide the complex mechanisms of execution behind the familiar concepts of a domain. The metamodels provide a foundation on which tools can be based, allowing new tools to be developed and deployed quickly.

Practice Risks

Tool support makes some tasks more convenient and others (such as extensive unit testing or CM) practical where they otherwise would not be. Inadequate or inappropriate tool support has a devastating effect on productivity in all parts of a project where tooling could be applied. Specific tool support risks include the following [Bass 2000a]:

lack of product-line-appropriate tools: The lack of true support for product lines often results in frustration at best and longer development cycles at worst. For example, if tools cannot coherently represent information about multiple products (especially their commonalities and variabilities), the entire concept of creating multiple products from the same core asset base is undermined. Users may be forced to extend existing tools until they become a heavy maintenance burden. New releases of a tool may invalidate the previous extensions, thereby risking time-consuming rework that will further lengthen the development cycle.
lack of support for variability: Controlled variability is essential for creating a product line that is flexible enough to accommodate changing customer needs. If your tools cannot help you represent and reason about variability early in the life cycle, you may not be able to
- express variability in the architecture, particularly in graphical form
- trace requirements variability to architecture variability
- trace component variability to architecture variability
- trace testing variability to component variability
lack of appropriate CM tool support: Product lines require sophisticated CM practices and robust tool support. Insufficient CM support (especially for capturing and correlating data about changes across multiple products) will chip away at an organization's ability to field products quickly and respond to requests for changes. Releases of the core assets that are not synchronized with product releases are another source of disruption to the overall product development schedule.
lack of tools supporting specific practice areas: If adopted tools fail to support adopted practices, the effort required to perform the latter may be excessive.
incompatibility between tools and practices: If an organization chooses tools and practices separately, their use may conflict, resulting in duplicated effort or failure to use the tools or to carry out the practices properly.
incompatibility between core asset development and product development: If product development uses tools or practices that are different from what core asset development anticipates, the provided core assets may not work well with them.
lack of interoperability: If the selected tools will not interoperate, the result will be inconsistencies and gaps between different product line activities.
lack of adoption: The tools will fail to be adopted by the relevant users. As a result, a heavy investment may be made in tools with little payback.
inappropriate use of automated derivation: If the automated modeling support and/or the domains associated with the product line are not sufficiently mature, the strategy for automated derivation may fail to efficiently yield products within the agreed-upon scope.
insufficient attention to detail in model building: If the organization lacks the discipline to define models completely (to the depth required by the tool chain), the resulting products may not operate as anticipated, if at all.