Software Engineering Institute

The SEI’s mission, as the nation’s only federally funded research and development center dedicated to software, is to establish and advance software as a strategic advantage for national security. On behalf of the U.S. Department of Defense (DoD), we are conducting research and development on generative AI not simply because it is software, but because it affects how our nation makes software, secures its cyber systems, and grows AI overall.

Generative AI is one of several emerging technologies that we enable the DoD to harness in a trustworthy, responsible, timely, and affordable manner. Last year we conducted a foundational study describing and categorizing exemplar LLM use case archetypes, concerns, and remedies. The resulting paper, Assessing Opportunities for LLMs in Software Engineering and Acquisition, provides a novel, structured way for decision makers in organizations with large software operations to assess the fitness of LLMs for addressing software engineering and acquisition needs.

Another SEI study, Demonstrating the Practical Utility and Limitations of ChatGPT Through Case Studies, explores how generative AI might enhance data science processes, training and education, literature reviews, and organizational strategic planning. Some of the researchers also experimented with ChatGPT to detect malicious code and vulnerabilities in source code, simulated realistic network activity using LLM directives, and explored deepfake video detection.

The researchers of the SEI project A Retrospective in Engineering Large Language Models for National Security established a test LLM to investigate the feasibility, cost, and trustworthiness of using generative AI in highly sensitive intelligence environments. The SEI also hosted a workshop that discussed DoD use cases, challenges, and needs for generative AI.

In our collective excitement to create new AI tools, generative or not, we must continue to maintain discipline at the end of the development lifecycle. Testing and evaluation remain essential for the safe and effective deployment of AI in defense platforms, as exemplified in a DoD-sponsored study of the Department of the Air Force that I had the honor to co-chair. The SEI is also researching rapid assurance of large-scale software systems, which can address potential new risks to system integrity and availability introduced by AI components. These efforts are steps on the roadmap laid out in the SEI’s National Agenda for Software Engineering Research and Development, particularly in three of its focus areas: AI-augmented software development, assuring continuously evolving systems, and engineering AI-enabled software systems.

Throughout even the most narrowly scoped studies, the SEI keeps the big picture in mind. Our ongoing work in AI engineering and responsible AI will continue to inform our generative AI research. Blog posts, podcasts, and live webcasts from our experts have offered high-level takes on LLMs: hype versus reality in software engineering and development, harnessing their power for economic and social good, leveraging tools, and critically assessing outputs.

It is still early days for generative AI, and its place in our lives will change as this technology rapidly evolves. But it certainly has a role in maintaining our nation’s strategic advantages in security and the global economy. As the national security enterprise is realizing generative AI’s potential to transform its mission, the SEI’s dedicated researchers are approaching generative AI with the same rigor, expertise, and collaboration we have brought to software engineering and cybersecurity for nearly 40 years.