Pedagogical Product Line
Overview
Business Case
Scope
Requirements
Concept of Operations
Architecture
Unit Test Plans
Production Plans
System Test Plans
Brickles Product
Pong Product
Bowling Product
Bibliography
Glossary
Misc Documents

Arcade Game Maker Pedagogical Product Line: System Test Plan Template

Overview

Identification

The Arcade Game Maker (AGM) product line organization will produce a series of arcade games ranging from low to high obstacle count with a range of interaction effects. For more details about AGM products, see Arcade Game Maker Pedagogical Product Line: Scope.

Document Map

The AGM product line is described in a series of documents that are related as shown in the following figure. This interactive map shows the order in which the documents should be read for the first time. After readers are familiar with the documents, they can go directly to the information needed.

This document is the system test plan template, which is based on the report by McGregor [McGregor 01b]. Product line organizations use it to capture how a new product is tested.


Document Map

Concepts

For definitions of basic concepts, see Arcade Game Maker Pedagogical Product Line: Acronym List/Glossary.

Readership

This document is intended primarily for product development teams. Managers can use it to determine what resources are needed to produce a product, and technical team members can use it to develop a detailed process for testing a product.

Plan

The AGM product line organization based its test plans on the Institute of Electrical and Electronics Engineers (IEEE) 829 standard. Each test plan will include the 15 sections listed below. For more details, see McGregor's report [McGregor 01b].

  1. Introduction
  2. Test Items
  3. Tested Features
  4. Features Not Tested (Per Cycle)
  5. Testing Strategy and Approach
    1. Syntax
    2. Description of Functionality
    3. Arguments for Tests
    4. Expected Output
    5. Specific Exclusions
    6. Dependencies
    7. Success/Failure Criteria for Test Cases
  6. Pass/Fail Criteria for the Complete Test Cycle
  7. Entrance and Exit Criteria
  8. Test Suspension Criteria and Resumption Requirements
  9. Test Deliverable and Status Communication Vehicles
  10. Testing Tasks
  11. Hardware and Software Requirements
  12. Problem Determination and Correction Responsibilities
  13. Staffing and Training Needs/Assignments
  14. Test Schedules
  15. Risks and Contingencies

Analyses

This section describes the techniques and agreed-upon standards used to test each component in the AGM product line. For justification and context, see the report by McGregor [McGregor 01b].

Coverage Standards

The system test coverage standard is based on use case information in Arcade Game Maker Pedagogical Product Line: Requirements Model. The frequency and criticality fields for each use case determine the correct level of test coverage. Both are rated on high, medium, and low scales. The following table shows the combinations of these values and the coverage standard defined for each.

Coverage Standards
Value Combination Frequency, Criticality Coverage Criteria Details
High, high Main X For each tested scenario, construct test cases for all pair-wise combinations of scenario variables that are defined in the system state machine.
Alternative X
Exceptional X
High, low Main X For each tested scenario, construct test cases for all pair-wise combinations of scenario variables that are defined in the system state machine.
Alternative X
Exceptional  
Medium, high Main X For each tested scenario, construct test cases for some pair-wise combinations of scenario variables that are defined in the system state machine.
Alternative  
Exceptional  
Medium, low Main X For each tested scenario, construct test cases for some pair-wise combinations of scenario variables that are defined in the system state machine.
Alternative  
Exceptional  

Examples

The analyses needed to support the creation of test cases from the use cases are conducted by filling out the next two tables. Example values are provided in the tables.

Data Type Analysis
Variable Data Type Equivalence Classes
age Integer 0 - 18; 19-21; 22-65; 66+
Test Case Variable 1 (age) Variable 2 Variable 3 Variable 4
1 12      
2 19      
3 56      
4 67      

Modifying the System Test Plan

Modify a specific unit test plan every time the unit being tested is changed using the techniques described in the previous section. Modify the generic unit test plan when it is shown that the techniques are not producing effective test cases and the standards are not producing satisfactory results. That modification process is shown in the following figure.


Attached Process

TE is computed for each unit test as shown below:

Identified defects are cataloged and then analyzed to determine their origin. At the end of a product line increment, the TE is computed on a component-by-component basis. When the average TE goes below 75%, make test coverage standards more comprehensive.

References and Further Reading

For details about the references cited in this document, see Arcade Game Maker Pedagogical Product Line: Bibliography.