Software Tool to Cut Cost of Static Analysis Adjudication and Code Repair
Created September 2024
This work leverages SEI’s expertise in automated code repair (ACR) and static analysis (SA) to decrease the cost of adjudicating SA alerts and repairing insecure code significantly and increase the trustworthiness of fielded software. While invaluable, SA tools can frustrate the separating of alert signal from noise to repair critical security weaknesses. This SEI research and development project produced a prototype tool, Redemption, to automatically repair code associated with SA alerts in security-relevant defect categories such as CERT C Coding Rules or MITRE Common Weakness Enumerations (CWEs).
SA Tools Produce Many False Positives, Lowering Tool Adoption
Developers using SA tools can build better quality software because those tools alert them to issues requiring repair. SEI analysis suggests that manually auditing and repairing SA alerts from one daily snapshot of the average mature codebase (approximately two million lines of C/C++ code) requires three-and-a-half person-years of effort. Because of that cost, some teams avoid using SA tools or reduce the scope of what those tools hunt for, even if urged to adopt SA technology.
In addition, recent advancements in ACR technology combined with analysis of a set of SA datasets suggest that many SA alerts in C/C++ could be repaired automatically, where determination of which repairs are needed can be made. For this work, the SEI targets categories of alerts that are repairable automatically—without requiring manual code audit or repair.
To qualify as repairable by the Redemption tool, a code defect must be able to be fixed in a way that can be automatically implemented by a repair tool, which uses information from the code’s abstract syntax tree (AST) in which the repair is to be placed. For example, buffer overflows are often not repairable because there is no automatic way to determine buffer capacity, but null pointer dereferences can be repaired by a test to compare the pointer with NULL.
Redemption Prototype Tool Automatically Repairs Three CERT C Coding Standard Violations
To estimate the impact of this project, SEI researchers analyzed C/C++ codebases for which it ran SA tools as part of SCALe audits. SCALe is a framework of tools and processes that aggregates output from SA tools and maps alerts to taxonomies of code flaws including CERT Secure Coding Rules and the Common Weakness Enumeration (CWE).
These SCALe-analyzed codebases totaled 233,900 lines of significant code on which 85,268 alerts were reported that allegedly violated 124 distinct CERT Secure Coding C or C++ rules. SEI discovered that 57,922 alerts (67.9% of the total alerts generated) violated just eight CERT rules. If 80 percent of the violations of those eight rules were fixed automatically, then 54.3% of all alerts would be addressed.
In response, the SEI team produced the Redemption tool to make automated repairs to C/C++ source code based on alerts produced by SA tools. The primary purpose of the Redemption tool is to improve the reliability of C/C++ source code quickly and cheaply. It handles false positives by repairing them as if there were true positives.
As of May 2024, the Redemption tool automatically repairs three types of CERT C Coding Standard violations:
The Redemption code repair system can be used as a command-line tool or through a script for use in a continuous integration (CI) development system. Through widely available tools, users can review and accept any repairs.
Related Areas of Research
This research work not only builds on SEI expertise in SA and ACR, but also aligns with the SEI objective to produce software that is trustworthy in construction and implementation and resilient in the face of operational uncertainties including known and yet unseen adversary capabilities. The work is an outgrowth of the SEI’s focus on the secure development of tools to promote better software through secure coding practices. It also connects with cybersecurity engineering where those involved with acquisition and development need to implement effective and repeatable practices to discover software vulnerabilities before they field applications.
Learn More
- Redemption: A Prototype for Automated Repair of Static Analysis Alerts
- Automated Repair of Static Analysis Alerts
- Applied Automated Repair of Static Analysis Alerts
- Automated Code Repair to Ensure Memory Safety
- Release of SCAIFE System Version 2.0.0 Provides Support for Continuous-Integration (CI) Systems
- A Public Repository of Data for Static-Analysis Classification Research
- SCALe v. 3: Automated Classification and Advance Prioritization of Static Analysis Alerts
- CERT C++ Secure Coding Guidelines
Looking Ahead
Next steps we’d like to take developing the Redemption tool include adding support for additional static analysis tools, increasing the number of alert repair categories, enhancing Redemption’s capability to work on Windows programs, and integrating additional workforce tools including IDEs and CI pipelines. To join in this continuing work, please contact the SEI.