AI Trust Lab: Engineering for Trustworthy AI

Created October 2019 • Updated April 2024

The Software Engineering Institute’s Trust Lab advances trustworthy, human-centered, and responsible AI engineering practices. The lab accelerates collaboration: we work with other experts and with stakeholder organizations to improve human interactions with AI systems and to create frameworks, tools, and guidelines to improve human interactions with AI-enabled and autonomous systems for DoD mission success.

Through both research and customer engagements, we conduct research and create tools to support trustworthy, human-centered, and responsible AI engineering practices by exploring the integration of responsible AI (RAI), human-computer interaction (HCI), user experience (UX), human-machine teaming (HMT), information architecture (IA), and other related practices and methods.

AI Must Work with—and for—People

AI holds great promise to empower us with knowledge and augment our effectiveness, but with that promise come challenges and risks. For example, bias in datasets can amplify discrimination, shortcomings in transparency and explainability can lead to mis- or dis-use of AI, and systems may reveal private or protected information without proper permission . We can—and must—ensure that we keep humans safe and in control, particularly with regard to government and public sector applications that affect broad populations and operate in high-stakes contexts. How can AI development teams harness the power of AI systems and design them to be valuable to humans?

Trustworthy systems are built to fit with mission and user needs, use appropriate data, and are reliable, robust, and secure. The challenge of developing and designing trustworthy, human-centered AI systems is a key focus for the AI engineering discipline, and the SEI established the Trust Lab to advance practices in engineering for trustworthy AI.

Developing Measurements of Trustworthiness Through Collaboration and Research

The Trust Lab collaborates with organizations, including the U.S. Defense Innovation Unit, the U.S. Chief Data and Artificial Intelligence Office (CDAO) , DARPA, and NIST, to develop measurements of trustworthiness in a variety of areas:

Use case and usability

Transparency and explainability

Equity, justice, and fairness

Likelihood of failure

Trustworthiness often depends on the data powering the AI system and the outcomes for people who use or are affected by the system. The Trust Lab team is currently creating tools to support data inspection and to help ML engineers and development teams to assess datasets fully to determine appropriateness. The Trust Lab is also examining aspects of negative bias, impacts, harms, and risk that can be introduced by an AI system and mitigating these aspects when needed.

Software Engineering Institute