Story-1287 · pramaan-docs ADR-110 · flag-gated self-audit

Atlas Reflection — `AtlasReflect` walkthrough

Branch issue-1287-atlas-reflection · Story: pramaan-io/pramaan-spec#1287 · Provenance: pramaan-docs ADR-110 (accepted 2026-06-12)

🛰️ 1 · Satellite — why this exists

PRAMAAN is a legal product. The single failure that destroys a lawyer's trust fastest is a fabricated specific — Atlas inventing a clause number he never read or an interest rate counsel never gave. Atlas's own soul already calls this "worse than no answer." But a rule written in a prompt is only ever mostly obeyed: a one-shot draft can still slip a "8% p.a." into the relief while counsel gave no rate.

WHY a second pass beats a longer prompt: the naive fix is "tell the model harder." That doesn't work — the same call that broke the rule is the call you're trusting to catch itself. The fix is a separate, narrow call whose only job is to audit the first one against rules that already exist. One cheap call aimed squarely at the catastrophic, named failure — and, because this is a legal audit trail, it emits what it caught: "drafted X, self-caught Y, shipped Z."

✈️ 2 · Airplane — the shape

AtlasIntake → draft proposal (verdict = all_clear)
                   │
            AtlasReflect(input, draft)   ← ONE extra call, audit-only, flag-gated
                   │
        passed? ── yes ─→ ship the draft, no event
                   │
                   no ──→ ship `corrected` + emit turn.reflection (the issues) as provenance

It runs only on the final all_clear turn (a proposal exists) and only if the flag is on. Never on a clarification turn — that keeps it to exactly one extra call per matter. It adds no new rules: every check below maps to a rule already in atlas.baml.

Check (`rule`)	Catches
`fabricated_specific`	invented clause/section/rule/case number, rate, figure, or date counsel never gave
`missed_contradiction`	value vs relief figure (the 10× minor-units trap: `value_at_stake_minor / 100`), party order
`overwrote_raw_words`	draft claims to replace counsel's sacred raw text
`unflagged_assumption`	a default filled without a note
`ungrounded_claim`	a statement not traceable to the input

🔬 3 · Microscope — what changed, and WHY each piece

Change	Where	What & WHY
Soul: new types + function	`agents/atlas/baml_src/atlas.baml`	`AtlasReflectionIssue`, `AtlasReflection` (`passed` / `issues` / `corrected: AtlasProposalCore?`), and `AtlasReflect(input, draft) -> AtlasReflection` on `AtlasBedrockClient`, schema `atlas-reflect-output-v1`. Audit-only prompt — names match the actual proposal types (`AtlasProposalCore.refined.relief_sought`), not the ticket's illustrative ones.
Regenerated client	`agents/atlas/baml_client/`	`baml-cli generate --from agents/atlas/baml_src`, committed. `b.AtlasReflect` now callable.
The flag	`atlas.yaml` + `config.py`	`reflection.enabled: false` (default OFF). Resolved in `AtlasConfig.reflection_enabled` with env override `ATLAS_REFLECTION_ENABLED` — same env-over-yaml pattern as the model/provider knobs. WHY a flag: faith now, measurement once golden cases exist; retire if it doesn't earn its latency.
The call	`agent.py`	`run_reflection(...)` — builds the client registry, calls `b.AtlasReflect`, and on ANY error returns `None` so a failed audit NEVER blocks the matter (Atlas always ships the draft).
The wiring	`runtime.py`	`_maybe_reflect` gates on flag + final `all_clear` + proposal-present. On a failed audit it swaps the proposal for `corrected`, folds reflection tokens into the run totals (broker cost cap sees true cost), and emits one `turn.reflection` event.
New event builder	`events.py`	`turn.reflection` — additive to `agent_event_stream.v1`, carries `{passed, issues, applied_correction}`. Emitted ONLY on a failed audit, so the default stream is unchanged (ADR-110: brains emit honest events; provenance rides the stream).

The catastrophic case, caught

// BAML test (atlas.baml): counsel gave NO rate; draft pleads "interest at 8% p.a."
@@assert({{ this.passed == false }})
@@check(flags_fabrication,
  {{ this.issues|selectattr("rule","equalto","fabricated_specific")|list|length > 0 }})

4 · The single most important guarantee

Flag OFF = zero behaviour change
With reflection.enabled=false (default), _maybe_reflect returns before any call, no turn.reflection event is emitted, and the event sequence is byte-identical to pre-1287: run.started → turn.delta → model.usage → turn.completed → run.completed. Proven by test_flag_off_makes_no_reflection_call (asserts the spy is never called AND the type sequence is unchanged).

5 · Tests

Test	Proves
`AtlasReflectCatchesFabricatedRate` (BAML)	invented "8% p.a." with no counsel rate → `passed == false` + a `fabricated_specific` issue (live-model check, like the other `atlas.baml` tests)
`test_flag_off_makes_no_reflection_call`	flag OFF → no extra call, identical event stream + usage
`test_flag_on_failed_audit_applies_correction_and_emits_issues`	flag ON + failed audit → `corrected` reaches counsel (no "8%"), `turn.reflection` carries the issues, `applied_correction=true`
`test_flag_on_passed_audit_is_quiet`	clean draft → no event, draft unchanged
`test_flag_on_skips_clarification_turn`	never runs when there is no proposal (clarification turn)

6 · Verification

$ (cd agents/atlas && pytest tests -q)
42 passed                      # 38 existing (zero regressions) + 4 new reflection tests
$ ruff check agents/
All checks passed!
$ baml-cli generate --from agents/atlas/baml_src
Wrote 14 files to agents/atlas/baml_client     # committed, reproducible (no diff on re-run)
# ADR-110 §6 statelessness: no sqlalchemy / psycopg / boto3 s3·sqs·dynamodb in the brain

7 · Honest caveat

No golden cases yet
We cannot yet prove reflection raises the pass rate — there is no golden set. The flag is precisely what lets it be A/B'd later and retired if the scoreboard says the latency isn't earned. The bet: one cheap call against a catastrophic, named failure. No new ADR — this enforces rules already written; the event-stream provenance is sanctioned by pramaan-docs ADR-110.

Atlas Reflection — AtlasReflect walkthrough