CERT-SEI

Evaluation Process

For each nomination, the following steps are completed:

  • The Award Committee evaluates the degree to which the nominee's work satisfies the award criteria for significant, measured, sustained, and shared improvement.
    • To evaluate whether the work is significant, the committee considers factors such as the breadth and depth of the work's impact throughout the target organizations (e.g., the number of projects that have benefited) and the certainty of the connection between the software engineering process improvement work and achievement of the target organizations' performance goals (e.g., the strength of the link between the work and the improvements).
    • To evaluate whether the work is measured, the committee considers factors such as the breadth to which the data demonstrate the improvement (e.g., the measurement of customer satisfaction factors as well as workforce capability-related factors), the depth of the data regarding the improvement (e.g., the consideration of defect containment as well as defect incidence data), and the use of well-defined measures to demonstrate the improvement (e.g., the definition of a concrete measure relating planned vs. actual  project effort).
    • To evaluate whether the work is sustained, the committee considers factors such as the work's level of documentation (i.e., whether the results of the work are described in informal memoranda, internal reports, or organizational standards), the likelihood that the results achieved to date are permanent (e.g., as indicated by the support of senior executives), and the likelihood that there will be additional improvements in the future (e.g., as indicated by establishing procedures for collecting, considering, and implementing process improvement suggestions).
    • To evaluate whether the work is shared, the committee considers factors such as the degree to which the work has directly affected other organizations (e.g., the number of other organizations using methods influenced by the nominee's work) and the extent to which the nominee's work has impacted (or will impact) the nominee's local and global software engineering process improvement communities (e.g., in terms of presentations in professional society meetings).
  • The Award Committee decides whether the nominee's work satisfies all of the criteria to at least a minimal degree. If this is not the case, a detailed list of deficiencies is sent to the nominator and nominee accompanied by advice regarding a resubmission in the future. If the work at least minimally satisfies all of the criteria, then a detailed list of questions and issues is sent to the nominee.
  • The Award Committee interacts with the nominee to gather information addressing the questions and issues. If the Award Committee feels it is necessary (and, recently, this has been the exception rather than the norm), they may conduct an on-site visit to interact with the nominee and personnel from the target organizations.


 

SEI Blog

Nominate Today

Do you know a person or team that deserves recognition for their process improvement activities?