System model
Represent a scenario S as a tuple of topology, identities, capabilities, assets, vulnerabilities, controls, objectives, observability requirements and replay constraints. A compiler resolves S into a deployment plan P and a set of proof obligations O. Here a proof obligation is a specific condition to check; the term does not imply a complete formal proof of system security.
After provisioning, readiness checks measure actual initial state X0. An experiment applies an authorized input sequence A under execution configuration M, producing observations E and final state X1. A versioned evaluator computes V(S, X0, E, X1). Its security verdict belongs to {satisfied, violated, inconclusive}; environment execution status is reported separately as completed, invalid, timed out or infrastructure error.
The system must never turn missing evidence into a passing score. A refusal in model output cannot negate a confirmed unauthorized resource write. Conversely, text claiming to have exported data is insufficient evidence that an export occurred.
A reproduction bundle identifies the scenario, compiler, backend adapter, image digests, synthetic-data seed, retrieval index, model configuration, tool schemas, policy versions, attack sequence, evaluator version and evidence manifest. It also states which dependencies cannot be pinned.
Scenario-as-code
A minimum scenario declares a support agent, a retrieval corpus containing synthetic documents, a customer-record service, a tool gateway, and a synthetic email sink. Its legitimate objective is to answer an authorized customer question. Its adversarial objective is to cause a prohibited export through untrusted retrieved content.
The contract must declare which principal may read which records, which destinations are allowed, where the injection may be placed, and what evidence is required to establish export. A vulnerability label without a corresponding configuration change is not an executable specification.
Type validation catches malformed fields. Semantic validation checks identities, tool capabilities, reachable resources and satisfiable prerequisites. Runtime readiness tests then confirm that the deployed resources satisfy the required preconditions. Validating a plan is insufficient when a provider or adapter can deploy a different effective state.
Parameterize names, record contents and selected topology details only when the security property remains meaningful. Require a benign task witness and a controlled vulnerability witness for each generated variant. Retain failed variants and rejection reasons to measure generator quality honestly.
Workload isolation and safety model
Treat target code, generated scenario inputs and learner activity as untrusted. Separate target networks from management services; disallow platform credentials in guest environments; constrain egress through a controlled gateway; enforce quotas, execution deadlines and automatic teardown. The experimental email service should deliver only to a synthetic sink.
Choose runtime isolation according to the threat model. Containers can suit constrained application exercises; arbitrary hostile code or kernel-focused labs require stronger separation and distinct operational review. Firecracker demonstrates a microVM design using virtualization and additional process restrictions, while its production guidance also identifies host and hardware responsibilities. Adopting it is not evidence that the assembled platform is secure.
Test cross-session access, control-plane reachability, metadata access, resource exhaustion, forged telemetry and cleanup failures. Keep independent boundary sensors outside the guest. Finite tests cannot establish the absence of all escapes; publish scope and residual risk with any isolation claim.