In late August 2026, the Army’s chief of Network Command (NETCOM) announced that to take full advantage of artificial intelligence (AI), the Army needed a comprehensive simulation of its networks. While acknowledging the difficulty in building a digital twin, Maj. Gen. Jacqueline Denise McPhail stated in Breaking Defense that a simulation would create “an ultra-realistic training ground for network operators, cyber defenders, and AI algorithms on how the network works and how to keep it working, even when under attack.”

Since 2017, the SEI has been developing and expanding the GHOSTS framework as a platform for simulating complex human, organizational, and machine behavior for cyber-training simulations. It began with simulated users operating real applications on hosts, producing the traffic, logs, files, browser history, and other artifacts defenders expect to encounter. It has since expanded into social and cognitive simulation, where actors communicate, form relationships, hold knowledge and beliefs, make decisions, and affect one another across a scenario—including through social influence, information propagation, trust, and the ways those interactions can change behavior and decision-making over time.

GHOSTS does not attempt to reproduce an entire operational environment. It models the parts needed to support a particular experiment, exercise, or mission rehearsal. A complete representation is expensive to build, difficult to validate, and often unnecessary. At the same time, omitting the wrong thing can undermine the event even when everything else is modeled well.

Digital twins re-expose an enduring simulation-design problem: How much of the operational environment actually needs to be reproduced? Every added system, behavior, and dependency increases cost and complexity. GHOSTS approaches the problem from the other direction, focusing on the human and organizational activity needed to make mission-relevant parts of the environment behave realistically. The question is not whether a simulation perfectly duplicates reality, but which parts of reality must hold up for the mission at hand. This is analogous, in the context of test and evaluation, to the concept of mission assurance.

This blog post, adapted from a recently published report, layers a philosophical perspective onto the engineering problems of sufficient realism in cyber simulations and proposes a five-step framework for determining the needed realism of a simulated environment.

Defining Realism

A common question we encounter among our mission partners is How do you know what is realistic?

More generally, do we need to accurately model all aspects of the real operating environment? For years, CERT has operated simulated environments for training, exercise, war gaming, and mission rehearsal, and keeps arriving at the same conclusion: Realism is tied to exercise validity and value. Dobson built that conclusion into the R-EACTR framework where the premise is that an exercise is valuable to the degree it feels real, and so realism belongs within every design decision that brings that exercise to life. The first GHOSTS technical report, published in 2018, confirmed that participants who rated the exercise more realistic also rated the training more valuable. GHOSTS grew from there as machinery to simulate a realistic environment inhabited by people performing activities that we would expect, given their role in an organization.

Out of this background and experience came sufficient realism. “Train as you fight” is a common phrase in military operations, and it sets a bar for training, exercise, and simulation. Otherwise, if the terrain, tools, or processes that a team trains with are significantly different from real-world operations, is it really a valid mission rehearsal? At the same time, cost, timing, and other constraints can often put full fidelity out of reach. Sufficient realism is the working answer to that tension—enough of the right factors in the environment, at the right fidelity, for the mission being rehearsed. Sufficient realism overlaps with many of the concerns captured by R-EACTR, and, in an exercise, resembles a reasonable and consistent storyline, realistic tactical maneuvers, defined rules of engagement, and authorization for action.

Military simulation has dealt with a closely related problem before. The SIMNET Program called its approach selective fidelity. Rather than reproduce a tank in complete physical detail, SIMNET began with the tasks soldiers needed to perform and included the controls, cues, and dynamics required to produce those training behaviors. Its designers could omit a functional bilge-pump switch because it contributed nothing to platoon- or company-level tactical training, while seemingly minor cues such as seat rumble and visual movement mattered because crews used them to judge speed and terrain.

Sufficient realism is rooted in the same spirit, but the cyber environment adds a complication. Some things have to be realistic even when they are not directly part of the training objective because they form the background against which the mission-relevant activity is interpreted. A defender may never be asked to evaluate ordinary employee web browsing, for example, but if all simulated users visit the same few sites, follow no deep links, produce almost no POST traffic, and begin work at the same instant, the background population becomes identifiable as synthetic. Once participants can separate simulation artifacts from ordinary activity, the exercise itself changes.

Selective fidelity tells us something important about where to spend fidelity, while sufficient realism also asks what minimum conditions the surrounding environment must satisfy before those selective choices remain valid. Above that baseline, mission analysis again determines where additional fidelity earns its cost.

Selective realism describes what the engineering approach does by narrowing the environment to the parts that need fidelity for the mission, while preserving enough background realism for those parts to be interpreted correctly. We use the benign term “taste” to describe how experienced people make difficult choices inside that approach when analysis does not uniquely determine an answer.

Mission analysis can tell us which systems, behaviors, relationships, and signals matter. It can constrain the range of acceptable models. But it does not always tell us whether a particular behavior is believable enough, whether a discrepancy is consequential, or when another increment of fidelity stops improving the rehearsal.

A longer answer to the realism question runs through our recent technical report, A Decade of GHOSTS in the Machine, and is woven into the framework's evolution from host-based activity generation through cyber-social-cognitive mission rehearsal. This post takes up one part of that answer in more detail: Taste.

Judgment Without a Formula

Taste can enter the picture only after the engineering problem has been narrowed. It is not a substitute for mission analysis, measurement, or validation. Selective realism and mission-focused modeling establish what parts of the operational environment matter, what operators need to observe and act on, and where fidelity should be concentrated. Taste is the trained judgment used within that constrained space when analysis does not fully determine whether a behavior is believable enough, whether a discrepancy matters, or when additional fidelity stops improving the simulation.

Questions about “good enough” are much older than simulation. Philosophers of science have approached different versions of the same problem: How do we judge something that cannot be reduced to a rule?

Engineers might object that if a claim rests on taste, which is seemingly subjective, then it cannot be proven wrong. This touches directly on the demarcation problem in the philosophy of science: What separates scientific claims from non-scientific ones? Karl Popper argued that a claim must specify the conditions under which it could be refuted. Philosophers continue to debate where to draw the line of what is science, but falsifiability gives us a useful standard of accountability here.

Other fields have had to make consequential decisions under similar conditions. RAND developed the Delphi method for defense forecasting under uncertainty. Nuclear risk assessment formalized expert elicitation because reactor failure probabilities cannot simply be established from observed frequency. The probabilistic risk assessments behind NUREG-1150, a risk assessment for accidents at five U.S. nuclear power plants by the Nuclear Regulatory Commission, depended on structured expert judgment. The intelligence community built structured analytic techniques for similar reasons, and Heuer's treatment of them remains standard training material. Naturalistic decision research likewise grew out of studying how fire commanders and military officers decide under time pressure.

The practical answer is to make judgment accountable. Identify who is qualified to make the call, capture the judgment before the outcome is known, and then compare it with the results.

The Very Old Problem of “Good Enough”

Aspects of simulation realism can be measured—traffic distributions can be compared against real captures, and classifiers can distinguish synthetic hosts from actual ones. Yet data alone cannot set the threshold or select which metrics matter for a mission.

Mission analysis narrows the problem considerably: What must participants observe, decide, and do? Which parts of the environment can affect those decisions? What surrounding conditions must remain credible for those observations to be meaningful?

Even after answering those questions, some choices remain. How good is good enough? Judged by whom? When does another increment of fidelity stop improving the rehearsal?

This is a long-standing philosophical question, and there are useful philosophical insights:

Aristotle identified this challenge in the Nicomachean Ethics, observing that an educated person demands precision only as far as the subject permits. Applied to simulation, fidelity must be tailored to the mission. Aristotle argues that over-engineering precision is a design flaw rather than rigor. While his principle is an important part of our argument, it also risks allowing a discipline to avoid measurement entirely if it defines its own limits—a tension we address below.

In the Poetics, Aristotle further notes that probable impossibilities are preferable to improbable possibilities. Applied to simulation, this point suggests that an approximation that preserves the cues participants need to reason correctly may be more useful than literal fidelity to details that do not affect the mission.

Plato makes a similar distinction in the Sophist, contrasting arts that copy exact proportions with sculptors who intentionally distort colossal statues, so they look correct from the viewer's perspective. In our experience, Plato’s distinction is a useful way to think about fidelity from the participant’s point of view.

To navigate these decisions, Aristotle relies on phronesis—practical wisdom grounded in experience and judgment rather than rigid rules. Gadamer later showed this capacity evolved into the modern concept of taste, a central theme for Enlightenment thinkers. Though formalized centuries later, the core problem of evaluating “good enough” remains unchanged. We see it today whenever a mission partner asks for a digital twin, a realistic cyber range, or a simulation of human behavior. No environment can reproduce every detail of the operational world, nor would every detail contribute equally to the mission. Designers still decide which features participants must be able to reason about, how faithfully those features must behave, and where additional realism does not improve the rehearsal.

Qualified Judges

A cyber range has a particularly unforgiving audience. Its users are actively looking for the edges and the seams. Unlike an audience willing to extend a suspension of disbelief, participants stake real operational judgment on the environment's accuracy. Because a false impression of realism carries real-world resources and costs, the environment has to prove itself to them. Sufficient realism should hold up under that scrutiny.

A recent AI cyber-range incident illustrates this problem from another direction. In a recent evaluation, a configuration error gave an OpenAI model access to the public internet even though the model had been told that internet access was not available. A fictional target also happened to correspond to a real domain, which the model then attacked as though it was within the simulated environment. The model reasoned from the environment it observed, rather than the intended environment. In this sense, realism includes boundaries as well as traffic, systems, and behavior. So, if the simulated world exposes a capability, participants can treat that capability as part of the scenario.

The first mission point here is that realism should be judged by people who know what the operational environment actually looks like.

A design team can determine whether the simulation behaves as intended. That is different from determining whether the behavior would survive scrutiny from someone who works in the environment being simulated. The people making the second judgment need experience with the real thing and enough comparative experience to recognize when something is off.

There are useful philosophical insights here as well:

David Hume gives us useful language for this. In Of the Standard of Taste, he grounds taste in qualified judges—those with “strong sense, united to delicate sentiment, improved by practice, perfected by comparison”—and illustrates these with the story of Sancho's kinsmen. Two tasters sample wine. One detects a faint taste of leather, the other of iron, and the rest of the room ridicules them both. But when the barrel is drained, an old key on a strip of leather lies at the bottom.

In this story, Hume specifies selection criteria for a judge. Cooke later developed formal methods for calibrating and weighing expert judgments. The intelligence community codified this as analytic tradecraft standards. The analogy to cyber exercise simulation is fitting. The analyst who says network traffic feels wrong while every dashboard reads green is Sancho's kinsman. The drained barrel is telemetry. The key on the leather is the beacon firing on a metronome, the scripted user-agent string, the browsing history with no deep links—whatever artifact turns up, after the fact and in the logs, to prove a hunch right that no rule could have produced in the moment.

The second mission point is that expert judgment has to leave evidence behind.

The initial hunch cannot be the end of the analysis. If the experienced analyst thinks the traffic is wrong, the range should preserve enough evidence to determine why. So, taste here is making claims that can later be tested. Hume's qualified judge turns out to look a lot like an experienced blue teamer staring at network traffic and saying, “Something about this is wrong.”

Additionally, Immanuel Kant noticed that these kinds of judgments are subjective but also come with an expectation of agreement—call something beautiful and you expect other practiced observers to concur, before any proof. Simulation judgments work similarly. When a designer says the browsing behavior looks wrong, they expect every practiced eye to see it too. The claim has the force of an example, and in the cyber domain, with artifacts, logs, and network traffic, we can verify.

The third mission point is that this kind of judgment has to be developed and preserved, precisely because much of it cannot be transferred through a rulebook.

Someone who has spent years looking at enterprise traffic acquires a sense for timing, density, application behavior, user rhythms, and other patterns that may be difficult to enumerate in advance. That expertise matters when the simulator produces something technically permissible but operationally strange.

Polanyi gives us another piece of the problem: “We can know more than we can tell.” Connoisseurship—in wine, in radiology, in traffic analysis—travels through apprenticeship and experience. In this sense, our decade of open-source development has proceeded in a similar spirit. The issue threads on GitHub, the exercise after-action reports, and the behaviors built out of hundreds of exercise scenarios are the cases from which the sensibility can be read, taught, and criticized. While our documentation says what the system is and can do, it is the record of its being built that indicates the taste we have used over the years to build the platform.

Sontag's Synthesis

Susan Sontag synthesizes these ideas in Notes on ‘Camp’, offering an account of how judgment operates without rigid rules. Sontag argues that artistic sensibility applies equally to domains where practitioners evaluate quality faster than they can formalize why. Sontag notes that while taste lacks formal proofs, individual judgments reveal a coherent sensibility—one that risks being destroyed if forced into rigid systems.

We did not begin GHOSTS with a formal theory of sufficient realism. Instead, the concept emerged through years of pragmatically informed concrete modeling decisions. Many of its important design decisions focused on what not to simulate.

This underlying logic becomes clear in considering what we have chosen to omit in the course of development. Early attempts to mimic full user behavior overburdened range infrastructure, prompting GitHub discussions on which details were unnecessary to replicate. Wasting resources on fidelity that is imperceptible is poor design, and omitting this kind of work is an exercise in developer taste. The same reasoning shapes GHOSTS’ defaults, much like Hansson's omakase argument for having default configurations embody choices about what should work out of the box.

In cyber simulation, hardening taste into fixed rules carries operational risks beyond aesthetics. Formulaic realism creates predictable patterns that defenders can easily filter out—a flaw in early simulation tools. When a sensibility becomes a rigid checklist, we can look for that checklist and find its identifiable signature.

There is another problem: Taste does not transfer cleanly between domains. Someone who can spot synthetic network traffic immediately may have no comparable intuition for whether a simulated persona or trust relationship is plausible. As GHOSTS expands across cyber, social, and cognitive domains, realism therefore depends on multiple kinds of specialized judgment rather than one rigid rulebook.

A Five-Step Framework for Sufficiency

We can make this concept of judgment more reusable by asking five ordered questions. Note that starting with fidelity or tooling instead of purpose tends to produce a simulation built around what is possible rather than what the mission requires.

Stage five does much of the work here, and it is easy to skip. It relates to the concept of defeasible reasoning and the concept of defeaters used in safety and assurance cases. Rather than only collecting evidence that supports a claim, we also ask what evidence would undermine it. Here, the claim is that the simulated environment is sufficiently realistic for its intended purpose. The SEI's work in producing cyber exercises over the last decade has helped us develop a list of tells that indicate sufficient realism has not been attained, and would challenge our claim:

• Defenders find a filter that isolates synthetic traffic.
• General traffic patterns are too predictable.
• GET traffic overwhelms POST activity.
• Browsing patterns remain too narrow throughout the exercise.
• Timelines spike in ways no workday can produce.
• The environment fails to support the training objective.

Writing these down before the event—and instrumenting the environment to detect them—makes the after-action discussion much more concrete.

Three Tests for Making Judgment Accountable

Sufficient realism still faces an obvious problem: If it rests on judgment, how can the judgment be evaluated? We noted earlier that Aristotle's claim enables a field to escape measurement. A similar problem appears in model-based systems engineering (MBSE), where reasoning can begin with tacit models in the heads of designers, move through informal representations, and eventually reach formal models with explicit semantics. Reasoning is possible at each level, but assumptions become easier to inspect and challenge as the model becomes more explicit.

The same principle applies here. The way out of this circle is to make judgments explicit enough that parties outside the project can test them. The following three tests are available:

• First, the claim that realism judgment resists codification can be refuted by codifying it. If a rule set identifies problems in a simulated environment more reliably than experienced reviewers do, the rule set should be adopted. We would surely adopt it as well.
• Second, the party evaluating whether an environment is believable should be independent of the team that built it. Stage 4 of the framework exists for this reason.
• Third, disagreement among qualified reviewers can be measured. When practiced analysts working from the same evidence continue to reach different conclusions, the limit of available precision is real and its location is identifiable. When they converge, additional precision was available and the design did not pursue it. Inter-rater agreement measures this directly.

The fifth stage makes falsification practical: State in advance what would indicate that the environment is insufficient, conduct the exercise, and determine which of those statements survive contact with participants. This is the ordinary test of an empirical claim, and it requires no modification to apply here.

The 2018 GHOSTS report found that higher perceived realism correlates with greater training value—a measurable construct that has seen improved instrumentation.

What we get is a standard of accountability rather than formal proof. A claim about sufficiency should be stated before the exercise, evaluated by qualified parties outside the design team, checked against recorded evidence, and revised when an exercise contradicts it. That standard is the same one Delphi, expert elicitation, and structured analytic techniques settled on, applied to a different subject.

Sontag's warning applies to the framework, in that a set of questions that we might harden into a compliance form would produce regularity that makes synthetic environments identifiable. The framework specifies the questions and requires that the answers be documented. The answers remain judgments.

A Note on Taste in the Age of AI

The AI space is changing quickly, and GHOSTS has had to change with it. Earlier, we built GHOSTS Shadows as a server-side way to manage LLM integration and connect model outputs to GHOSTS activity. That was useful, but it reflected an earlier stage of the problem—providing GHOSTS agents with access to locally hosted LLMs, particularly inside closed or air-gapped environments. RangerAI is becoming our AI infrastructure stack around our cyber range and simulation work, the place where planning, model integration, orchestration, tool use, and exercise-level AI control can live. In that model, GHOSTS does not need to own the entire AI layer.

GHOSTS remains the realism and actuation engine, the system that makes synthetic users, teammates, organizations, and public actors behave inside the simulated world. RangerAI can handle more of the AI infrastructure, which continues to evolve quite rapidly. As that ecosystem matures, GHOSTS remains focused on its core mission.

AI also changes what adversaries and defenders can do inside the simulated world. It can increase the scale and variation of content generation, adapt actions based on scenario state, support richer NPC behavior, accelerate scenario construction, and help operators explore alternate paths.

The power of AI is most useful when connected to a disciplined simulation architecture including actor models, scenario constraints, workflow logic, telemetry, repeatability, and after-action review.

For GHOSTS, the main AI question is not whether a model can generate a plausible sentence or activity directive. Instead, we are concerned with how a generated action fits into a simulation. What state did the model see? Why did the NPC act? What changed afterward? Was the action executed? Was it observed? Did it affect another NPC, a defender, a public narrative, or a mission outcome? By focusing on these impacts, we work to use AI output in a methodical, focused manner. In other words, with AI we have a more responsive and powerful approach to experimentation in support of our choices regarding taste—what constitutes sufficient realism and how to validate the modeling that is done.

The harder problem remains. AI can generate more varied behavior, adapt NPC actions to changing conditions, and help simulations explore many more possible scenarios, but scale does not answer the question of what realism the mission requires. As GHOSTS expands across cyber, social, and cognitive environments, sufficient realism will depend on understanding what state an actor can observe, why it acts, what changes as a result, and whether those effects matter to the specific mission being rehearsed. Future work will therefore be less about making simulations ever more complete and more about developing, testing, and transmitting the judgment needed to decide what must be realistic, for whom, and to what end.