Native AI for MBSE: Lessons Learned and Measures from a Knowledge-Grounded Experimentation Platform

This presentation, delivered at the AI4SE & SE4AI Workshop 2026 on September 23, 2026, shares insights from a knowledge-grounded experimentation platform for native AI integration in model-based systems engineering. It evaluates the frontier models GPT 5.6 Sol and Claude Opus 5 on standardized SysML v2 modeling tasks under three conditions: unassisted generation, SysML v2 language validation, and a full toolchain combining validation with a Model Context Protocol (MCP) server, a topic-organized markdown knowledge base, and agentic workflow skills.

The presentation explained the benchmarking workflow and a knowledge pattern measure that assesses observable modeling breadth and depth across requirements and traceability, structure, behavior, analysis, verification, and organization and views. The lessons show how validation removes broad classes of language errors, while knowledge retrieval and workflow guidance strengthen modeling depth and consistency. Additional takeaways address topic-level knowledge gaps, repeatable model and tool comparisons, token and time tradeoffs, and why pattern evidence should not be interpreted as proof of semantic correctness.