2022 Year in Review
Juneberry Version 0.5 Simulates Attacks on Machine Learning Systems
Juneberry is an open source Python tool, created by the SEI’s Artificial Intelligence (AI) Division, that verifies and validates machine-learning (ML) models by automatically training, evaluating, and comparing them against multiple data sets. The latest version of Juneberry adds the ability to simulate attacks that could disrupt computer vision systems.
Very specialized semantics give users the knowledge of how these attacks are structured. These semantics are not available elsewhere.
Andrew MellingerPrincipal Engineer, SEI AI Division
Property inference attacks reveal information about an ML system’s training data. Using that information, attackers can craft and inject watermarks into image data to trick the ML system into unintended behavior.
Juneberry 0.5 allows users to prescribe ML model changes that simulate property inference attacks. “Very specialized semantics give users the knowledge of how these attacks are structured,” said principal engineer Andrew Mellinger. “These semantics are not available elsewhere.”
The attack simulation capability will allow ML developers and researchers to test their models for weaknesses against property inference attacks and ultimately protect ML-powered image classification systems.
Explore Juneberry at https://github.com/cmu-sei/juneberry.
RESEARCHERS
Andrew Mellinger (project lead), Bryan Brown, Matthew Churilla, Shannon Gallagher, Jon Helland, Daniel Justice, Dominique Mittermeier, Hayden Moore, William R. Nichols, William Shaw, Anusha Sinha, Nathan VanHoudnos, Jordan Widjaja, Nick Winski, John Zucca
Mentioned in this Article
Property inference attack configuration for Juneberry
