Arcade Game Maker Pedagogical Product Line: System Test Plan Template
The Arcade Game Maker (AGM) product line organization will produce a series of arcade games ranging from low to high obstacle count with a range of interaction effects. For more details about AGM products, see Arcade Game Maker Pedagogical Product Line: Scope.
The AGM product line is described in a series of documents that are related as shown in the following figure. This interactive map shows the order in which the documents should be read for the first time. After readers are familiar with the documents, they can go directly to the information needed.
This document is the system test plan template, which is based on the report by McGregor [McGregor 01b]. Product line organizations use it to capture how a new product is tested.
For definitions of basic concepts, see Arcade Game Maker Pedagogical Product Line: Acronym List/Glossary.
This document is intended primarily for product development teams. Managers can use it to determine what resources are needed to produce a product, and technical team members can use it to develop a detailed process for testing a product.
The AGM product line organization based its test plans on the Institute of Electrical and Electronics Engineers (IEEE) 829 standard. Each test plan will include the 15 sections listed below. For more details, see McGregor's report [McGregor 01b].
- Test Items
- Tested Features
- Features Not Tested (Per Cycle)
- Testing Strategy and Approach
- Description of Functionality
- Arguments for Tests
- Expected Output
- Specific Exclusions
- Success/Failure Criteria for Test Cases
- Pass/Fail Criteria for the Complete Test Cycle
- Entrance and Exit Criteria
- Test Suspension Criteria and Resumption Requirements
- Test Deliverable and Status Communication Vehicles
- Testing Tasks
- Hardware and Software Requirements
- Problem Determination and Correction Responsibilities
- Staffing and Training Needs/Assignments
- Test Schedules
- Risks and Contingencies
This section describes the techniques and agreed-upon standards used to test each component in the AGM product line. For justification and context, see the report by McGregor [McGregor 01b].
The system test coverage standard is based on use case information in Arcade Game Maker Pedagogical Product Line: Requirements Model. The frequency and criticality fields for each use case determine the correct level of test coverage. Both are rated on high, medium, and low scales. The following table shows the combinations of these values and the coverage standard defined for each.
|Value Combination Frequency, Criticality||Coverage Criteria||Details|
|High, high||Main||X||For each tested scenario, construct test cases for all pair-wise combinations of scenario variables that are defined in the system state machine.|
|High, low||Main||X||For each tested scenario, construct test cases for all pair-wise combinations of scenario variables that are defined in the system state machine.|
|Medium, high||Main||X||For each tested scenario, construct test cases for some pair-wise combinations of scenario variables that are defined in the system state machine.|
|Medium, low||Main||X||For each tested scenario, construct test cases for some pair-wise combinations of scenario variables that are defined in the system state machine.|
The analyses needed to support the creation of test cases from the use cases are conducted by filling out the next two tables. Example values are provided in the tables.
|Data Type Analysis|
|Variable||Data Type||Equivalence Classes|
|age||Integer||0 - 18; 19-21; 22-65; 66+|
|Test Case||Variable 1 (age)||Variable 2||Variable 3||Variable 4|
Modifying the System Test Plan
Modify a specific unit test plan every time the unit being tested is changed using the techniques described in the previous section. Modify the generic unit test plan when it is shown that the techniques are not producing effective test cases and the standards are not producing satisfactory results. That modification process is shown in the following figure.
TE is computed for each unit test as shown below:
Identified defects are cataloged and then analyzed to determine their origin. At the end of a product line increment, the TE is computed on a component-by-component basis. When the average TE goes below 75%, make test coverage standards more comprehensive.
References and Further Reading
For details about the references cited in this document, see Arcade Game Maker Pedagogical Product Line: Bibliography.