Improve Your AI Classifiers with AIR Using Causal Discovery, Identification, and Estimation

The Department of Defense (DoD) is increasing its use of artificial intelligence (AI) classifiers and predictors; however, users may grow to distrust the results because AI classifiers are subject to a lack of robustness (i.e., the ability to perform accurately in unusual or changing contexts). Drift in data/concept, evolving edge cases, and emerging phenomena undermine the correlations relied upon by AI. New test and evaluation methods are therefore needed for ongoing evaluation. The SEI AIR tool offers a precedent-setting capability to improve the correctness of AI classifications and predictions, increasing confidence in the use of AI in development, testing, and operations decision-making.

We are seeking DoD collaborators to use and provide feedback on our technology. As a participant, your AI and subject-matter experts will work with our team to identify known causal relationships and build an initial causal graph. Our process involves using a cutting-edge causal discovery tool, Tetrad, with custom causal identification algorithms and stacked super-learners using doubly robust causal estimators to build an AI “health report.” Your report will include a confidence range of expected treatment effects from your data and interpretations of the causal graph to give you actionable insights into your AI classifier’s health.