Automating Vulnerability Discovery in Critical Applications
Created September 2017
Vulnerabilities are too pervasive in software-based systems to find them all manually. CERT researchers develop automated tools that discover and mitigate software vulnerabilities and transfer them to security researchers, procurement specialists, and software vendors.
Automating Vulnerability Discovery
There are many places in the software lifecycle where software vulnerabilities can be discovered and mitigated. We develop new automated tools and techniques and put them in the hands of security researchers, procurement specialists, and software vendors to help them improve and evaluate the security of the software ecosystem used by the U.S. Department of Defense (DoD) and the U.S. government (USG).
These tools and techniques are intended to be used by vendors during software development. Still, vulnerabilities exist in delivered software. In a project called “Automating Vulnerability Discovery,” we focused on the ability to automatically discover vulnerabilities after software is developed and shipped, without assistance from the software vendor. This approach makes the problem more challenging because we must analyze programs at the binary level. But this requirement is critical in practice because many vendors are reluctant to share access to their products’ source code, which they regard as sensitive and proprietary.
There Are Too Many Vulnerabilities Not to Automate
Vulnerabilities are pervasive in software-based systems and protocols, both in traditional IT networks and in those that support critical U.S. infrastructure. Software is written and updated frequently, and there are not enough human analysts to keep pace with the speed of production. Automation is the only way to keep up.
Led by Dr. Edward Schwarz, the Vulnerability Discovery project aims to reduce vulnerabilities in critical DoD and USG systems by improving techniques for automated vulnerability discovery. If vendors adopt our techniques in their software development processes, the DoD will acquire software applications that are hardened—and more secure—before and after they are deployed into the DoD infrastructure.
Our Collaborators
CERT researchers are working with the startup company ForAllSecure on a new technique for automating vulnerability discovery. We are also using research from the Carnegie Mellon University CyLab.
Our Approach: Automate Discovery, Prioritize Results, Address the Most Important Vulnerabilities First
Our approach to vulnerability discovery builds on an extensive body of work at Carnegie Mellon University and the SEI's CERT Division. This work includes the following techniques:
- Black-box mutational fuzzing—Mutational fuzzing randomly introduces mutations into starting inputs called seed files. It then runs these modified inputs on the program to see if it causes a crash. Fuzzing is one of the most effective vulnerability discovery techniques in practice, and the SEI maintains its own black-box fuzzer, the CERT Basic Fuzzing Framework.
- Vulnerability uniqueness determination—A primary challenge in vulnerability discovery is understanding which vulnerability triggered a crash and how serious that vulnerability is. For example, in one of our recent experiments, a major vendor's software produced over 40,000 exploitable crashes, but these crashes were caused by only a few vulnerabilities. We use a new technique that applies software patches to identify the scope of a vulnerability, which allows us to precisely count the number of vulnerabilities that a particular vulnerability discovery technique identifies.
Combining Fuzzing with Concolic Execution
ForAllSecure is a start-up company on a mission to make software safer. The ForAllSecure team builds software security tools for developers, enterprises, and end users that automatically find vulnerabilities in software at the binary level, using research transitioned from the Carnegie Mellon University CyLab.
ForAllSecure is one of the world’s leading experts on binary concolic execution and is widely known for its Mayhem concolic executor, which recently won DARPA’s Cyber Grand Challenge competition. Concolic execution is a powerful form of symbolic program analysis that describes program executions as logical formulas and solves them to trigger and test new fragments of program code. Concolic execution is known for its ability to trigger code that is difficult to reach, allowing it to find vulnerabilities that other techniques such as fuzzing might miss. This power comes with a price, however, as concolic execution tends to be slow and has not scaled to large programs.
By joining our expertise in fuzzing with ForAllSecure’s expertise in concolic execution, we have been collaborating on a new technique that brings the best of both fuzzing and concolic execution. Fuzzing brings scalability, speed, and the ability to discover vulnerabilities in large, complex programs. Concolic execution allows analysts to omit seed files, making the process of vulnerability discovery easier than ever. It also allows software analysts to test and detect vulnerabilities in code that is difficult for a fuzzer to reach.
Software and Tools
CERT Tapioca
July 2017
CERT Tapioca is a network-layer MITM proxy utility that checks for apps that fail to validate certificates and investigates content of network traffic, including HTTP and HTTPS.
downloadCERT Dranzer
June 2015
Dranzer is a tool that enables users to examine effective techniques for fuzz testing ActiveX controls.
downloadCERT Triage Tools
May 2014
CERT Triage Tools consist of a triage script and a GNU Debugger (GDB) extension named 'exploitable' that classify Linux application defects by severity.
downloadLearn More
Using Alternate Data Streams in the Collection and Exfiltration of Data
September 19, 2022 Blog Post
Dustin Updyke, Molly Jaconski
In this blog post, we describe how attackers obscure their activity via alternate data streams (ADSs) and how to defend against malware attacks that employ...
read
Designing Vultron: A Protocol for Multi-Party Coordinated Vulnerability Disclosure (MPCVD)
September 15, 2022 Special Report
Allen D. Householder
This report proposes a formal protocol specification for MPCVD to improve the interoperability of both CVD and MPCVD processes.
read
Coordinated Vulnerability Disclosure User Stories
August 25, 2022 White Paper
Brad RunyonEric HatlebackAllen D. Householder
This paper provides user stories to guide the development of a technical protocol and application programming interface for Coordinated Vulnerability Disclosure.
read
An Analysis of How Many Undiscovered Vulnerabilities Remain in Information Systems
March 09, 2022 White Paper
Jonathan Spring
This paper examines the paradigm that the number of undiscovered vulnerabilities is manageably small through the lens of mathematical concepts from the theory of computing.
read
VINCE: A Software Vulnerability Coordination Platform
January 25, 2021 Podcast
Emily SarnesoArt Manion
Emily Sarneso, the architect of VINCE, and Art Manion, technical manager of the Vulnerability Analysis Team in the SEI CERT Division, discuss the rollout of VINCE, how to use it, and future work in vulnerability coordination.
learn more
CERT/CC Releases VINCE Software Vulnerability Collaboration Platform
June 04, 2020 News
The web-based platform enables direct communication between vulnerability stakeholders.
read
Projecting How AI/ML Will Revolutionize Software Vulnerability Discovery
February 14, 2020 Video
Greg Shannon
Watch as Dr. Greg Shannon, SEI CERT Division Chief Scientist, moderates discussion on projecting how AI/ML will revolutionize software vulnerability discovery, mitigation, and/or coordination
watch
CERT Tapioca
July 12, 2017 Software
CERT Tapioca is a network-layer MITM proxy utility that checks for apps that fail to validate certificates and investigates content of network traffic, including HTTP and HTTPS.
download