Dynamic, randomized-input functional testing, or black-box fuzz testing, is an effective technique for finding security vulnerabilities in software applications. Parameters for an invocation of black-box fuzz testing generally include known-good input to use as a basis for randomization (i.e., a seed file) and a specification of how much of the seed file to randomize (i.e., the range).This report describes an algorithm that applies basic statistical theory to the parameter selection problem and automates selection of seed files and ranges. This algorithm was implemented in an open-source, file-interface testing tool and was used to find and mitigate vulnerabilities in several software applications. This report generalizes the parameter selection problem, explains the algorithm, and analyzes empirical data collected from the implementation. Results of using the algorithm show a marked improvement in the efficiency of discovering unique application errors over basic parameter selection techniques.
This report is related to the following area(s) of work:
Security and SurvivabilityTechnical Note
CMU/SEI-2012-TN-019
August 2012
SEI:
Householder, Allen; & Foote, Jonathan. Probability-Based Parameter Selection for Black-Box Fuzz Testing (CMU/SEI-2012-TN-019). Software Engineering Institute, Carnegie Mellon University, 2012. http://www.sei.cmu.edu/library/abstracts/reports/12tn019.cfm
IEEE:
A. Householder, and J. Foote, "Probability-Based Parameter Selection for Black-Box Fuzz Testing," Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania, Technical Note CMU/SEI-2012-TN-019, 2012. http://www.sei.cmu.edu/library/abstracts/reports/12tn019.cfm
APA:
Householder, A., & Foote, J. (2012). Probability-Based Parameter Selection for Black-Box Fuzz Testing (CMU/SEI-2012-TN-019). Retrieved May 22, 2013, from the Software Engineering Institute, Carnegie Mellon University website: http://www.sei.cmu.edu/library/abstracts/reports/12tn019.cfm
CHI:
Householder, Allen, and Jonathan Foote. Probability-Based Parameter Selection for Black-Box Fuzz Testing (CMU/SEI-2012-TN-019). Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University, 2012. http://www.sei.cmu.edu/library/abstracts/reports/12tn019.cfm
MLA:
Householder, A., & Foote, J. 2012. Probability-Based Parameter Selection for Black-Box Fuzz Testing (Technical Report CMU/SEI-2012-TN-019). Pittsburgh: Software Engineering Institute, Carnegie Mellon University. http://www.sei.cmu.edu/library/abstracts/reports/12tn019.cfm
For more information