Product Evaluation & Selection

Lesson ES1: Every off-the-shelf item used in the system should be subject to an appropriate evaluation and selection process.

The tools you choose to support your development and maintenance are as important as the products incorporated into your system for the end users. Your tools deserve consideration that is as careful as that given to what the end users receive.

Lesson ES2: A sound COTS product evaluation process must support the selection.

Most product selections are too important to leave to whim or politics, and laziness or lack of resources should not become the determining factor in how much evaluation is done. Engineering of a good COTS solution starts with careful selection.

Before you can figure out what products you need, you must have some notion of the job they are expected to do. As you learn more about the marketplace, that knowledge will play an increasing role in the formulation of criteria, and requirements will be tempered by what is possible.

Lesson ES4: Careful consideration must be given to the identification of selection criteria.

Selection criteria are your means of expressing what you are looking for from the marketplace. Many projects do not take enough time to develop a comprehensive and effective set of criteria.

Lesson ES5: Pilots and demonstrations are essential selection tools.

There is no substitute for hands-on examination of candidate COTS products. It is a major mitigation strategy for the risks associated with the use of COTS products.

Lesson ES6: Product and technology maturity must be considered.

An understanding of the marketplace and the technologies expressed in products is essential. That extends to information on how long the product or technology has been in use and where it is likely to go in the future.