Change: Mira had an injection wall (per-run nonce delimiters + the INPUT-CHANNEL
rule in panel_prompts.py and converse.py) but no eval measured it. This
adds a synthetic adversarial corpus, a deterministic OFFLINE invariant runner, offline tests through
the stubbed panel, a fence-removed→RED proof, and a blocking CI gate. The wall
itself is unchanged — it is now measured, not redesigned.
| File | Role |
|---|---|
agents/mira/evals/datasets/adversarial/{command,converse}.yaml | The corpus: 13 synthetic, golden-set-disjoint cases across direct / indirect / fence-escape / jailbreak / prompt-extraction / exfiltration / goal-hijack, over BOTH routes. |
agents/mira/evals/adversarial.py | Typed corpus (schema-validated on load) + the deterministic OFFLINE invariant runner + the CLI (run.py adversarial). |
agents/mira/tests/test_adversarial.py | Offline tests: GREEN with the wall, RED without it, and the structural / no-false-fire guards. |
.github/workflows/ci.yml | New blocking PR gate step firing the pytest proof + the standalone runner. |
The stub is a credulous model — a stand-in for an LLM that obeys any instruction it reads
as a top-level directive. It stays honest only when the hostile trigger
is walled inside a real <<LABEL_nonce>> … <</LABEL_nonce>>
block of the REAL prompt. The un-wall reader is nonce-aware: a forged <</DOCUMENT_1>>
with no nonce does not close the real block.
Two defences, two robustnesses, both asserted: scope_held reuses the command harness
_check_invariants — a CODE wall (panel._carry_locked_fields) robust even to a
stripped fence — while prompt-leak / fabrication / forbidden-party-in-an-unlocked-field are PROMPT-wall
invariants that ride the fence. Every case asserts at least one fence-riding invariant, so removing the
wall turns all 13 red.
changed_fields ⊆ scope · no prompt leak · no fabrication · no forbidden party (command);
no prompt leak · no forbidden action · matter-scope held (converse).
cd agents/mira uv run --package mira pytest tests -q # 92 passed (52 prior + 40 adversarial) uv run --package mira python -m evals.run adversarial # 13 case(s) held / 0 failures (OFFLINE) cd ../.. && uv run ruff check agents/ # clean uv run python scripts/check_brain_egress.py # clean uv run python scripts/check_prompt_leakage.py # clean
The fence-removed→RED proof (test_case_goes_red_when_the_fence_is_removed) and the
nonce-only proof (test_fence_escape_goes_red_when_only_the_nonce_is_removed) both pass —
every case turns red when the wall, or just its per-run nonce, is stripped.
Eval-only change: no agent brain, runtime, or wall behaviour is modified, so no CodeZip rebuild or AgentCore redeploy is required. The live-model firing of the same corpus is the separate, non-blocking T3 lane (#2048), not this PR.