Story 1609: The Dispatcher Who Hands Blake the File (the registrar broker-consumer, Blake-seam P3b)
Why this exists (the catastrophe it prevents). Story-1608 put a runner on the track: registration batons now get picked up and carried to the registrar queue. But a baton carried to a window where no one is standing is still a dropped baton. Until this Story, the registrar queue had no consumer: a message could land there and sit until it timed out into the DLQ, and Blake — the Record Registrar who names, types, and tags a freshly-indexed document — would never be invoked. Worse, this is the one hop where the trace could silently snap: there was nothing that built the agent_invoke.v1 envelope and stamped meta.traceparent onto it. Blake's far side was already wired to extract a traceparent from meta (so its spans nest under the upload's trace) — but it was waiting for a carrier no one was sending. This Story is the dispatcher at the window: it takes the registration request off the queue, pulls the document's markdown and the firm's catalog, hands Blake a complete dossier, listens to what Blake proposes, walks each proposal to the one office allowed to act on it (the broker executor), and finally stamps the document registered. And it stamps the trace carrier on the handoff, so the single trace_id that began at upload runs unbroken all the way into Blake's reasoning and back.
The one analogy: the records-room dispatcher
Blake is a brilliant classifier locked in a windowless room with no keys — no database password, no S3 credentials, no ability to touch a single record (pramaan-docs ADR-128 leash). He can only read what is slid under the door and write proposals on slips of paper that he slides back. The dispatcher (this consumer) is the only one with keys. For each request the dispatcher: (1) claims the file ("I've got this one" — queued→registering), (2) gathers the dossier Blake needs — the document's prose (document.md) and the existing vocabulary so Blake reuses "Contract" instead of coining "Agreement", (3) slides it under the door with a tracking slip stapled on (the traceparent), (4) reads back Blake's proposal slips (his tool_intents), (5) walks each slip to the broker executor — the only office that may actually write to a record, and only under auth + RLS + policy + idempotency + an audit trail — and (6) stamps the file's status with what Blake concluded (registered / needs_review / failed). The dispatcher never reasons about the document and never lets Blake touch a record; Blake never holds a key and never writes truth. That separation is the design.
derived/document.md; P2 (#429) gives the broker-side read_document_markdown; P3a (#426) drains the outbox onto the registrar queue. This Story consumes that queue. It needs Blake's AgentCore runtime ARN, which P4 publishes to SSM /pramaan/staging/agents/registrar-runtime-arn; the deploy workflow reads it into AGENT_RUNTIME_ARN_REGISTRAR. Earliest end-to-end milestone is P5: one upload → registered + a Blake proposal persisted + ONE continuous trace_id.๐ฐ๏ธ Satellite: what shipped
One new SQS-consumer worker, one settings field, two manual-span registry names, one deploy workflow. No routes, no DTO contract changes, no migration — the envelope, the executor, the registration state machine, the status-event ledger, and Blake's contracts all already exist — so no OpenAPI mirror moves and there is a single alembic head.
- The control plane is testable, the wiring is thin —
processor.pyholds the whole control plane behindRegistrarConsumerDeps(DB, AgentCore, S3, the runtime ARN);handler.pyis the thin Lambda that parents the per-record span and reports batch-item failures. The split mirrors the extraction worker exactly. - The intents live inside
turn.completed— Blake does not emit separatetool.requestedevents; it emits oneturn.completedwhoseturndict carries theverdictand atool_intentslist. The consumer folds the stream, then routes each intent throughexecute_tool_intent(the one path to business truth). - Verdict drives the terminal status —
all_clear→registered,needs_review→registration_needs_review,error→registration_failed; an unknown verdict fails toward human review, never a silent drop. - The trace is stamped on the handoff — under the approved
AGENT_INVOKE_BUILDspan the consumer injects the live W3C carrier intometa.traceparent(hop 5), so Blake'sagent_invoke_observability_context(hop 6) nests under the sametrace_id. The stored envelopetraceparentis the fallback so the chain survivesOTEL_SDK_DISABLED. - The deploy workflow — mirrors
deploy-document-extraction-worker.yml: an in-VPC Lambda with an SQS event-source mapping from the registrar queue, least-privilege IAM (S3 GetObject onderived/, SQS consume,bedrock-agentcore:InvokeAgentRuntime), and the two ESM preconditions baked in — queueVisibilityTimeout≥ function timeout, and a retry loop around the IAM-propagation race.
โ๏ธ Airplane: the eight hops, and where each lives
registrar SQS queue
โ (message body = the outbox envelope, traceparent stamped on it)
โผ
[hop 4] handler.lambda_handler โโ parents per-record span on envelope.traceparent
โผ claims the doc: queued โโถ registering (+status event)
[hop 4a] read_document_markdown (P2) + catalog pre-reads (existing types/tags/participants)
โผ
[hop 5] _invoke_brain โโ AGENT_INVOKE_BUILD: inject meta.traceparent
โ AGENT_INVOKE_CALL : invoke_agent_runtime(registrar ARN)
โผ
[hop 6] Blake (AgentCore) โโ agent_invoke_observability_context extracts meta.traceparent
โ โ every agents.registrar.* span nests under the SAME trace_id
โผ agent_event_stream.v1: run.started โ turn.completed{verdict, tool_intents} โ run.completed
[hop 7] consume_registration_stream โ execute_tool_intent per intent (broker executes; Blake never does)
โผ
[hop 8] RegistrationService.advance: registering โโถ registered | needs_review | failed (+status event)
๐ฌ Microscopic: the decisions that took judgment
Idempotency is a status gate, not a lock
SQS is at-least-once, so the same registration message can arrive twice. The consumer reads the document's registration_status first and routes on it: registered → no-op ("duplicate"); registering → re-drive in place (a prior attempt crashed mid-flight); registration_failed/registration_needs_review → requeue then re-pick (the ADR-128 ยง2 retry edges); queued → the normal claim. Beneath that, the effect executor's own (document_id, extraction_run_id) claim guarantees a re-run records 0 new effects, so even a duplicate that slips through writes no duplicate proposal. The anchor is run-independent on purpose: a redelivery under a fresh broker run_id still collides on the substrate, while a genuine re-extraction (new extraction_run_id) legitimately re-registers.
run.failed event, a stalled stream past the broker deadline, or an intent that raises) marks the document registration_failed in its own committed transaction and re-raises — so the handler reports a batch-item failure and SQS redelivers (visibility timeout → maxReceiveCount → DLQ), and the redelivery re-drives via the failed→queued retry edge once the outage clears. A clean error verdict (Blake honestly reporting it could not classify) is different: it is a terminal, acked outcome — registration_failed with no redrive storm. The dispatcher distinguishes "the window was closed" from "Blake read the file and said he can't name it."Why no user_id on the invoke
The canonical agent_invoke.v1 InvokeMeta requires a user_id because Atlas runs are counsel-launched through the WebSocket. Registration is event-triggered — no human launched it — so the consumer builds the envelope as the plain wire dict Blake's runtime actually reads (mode, meta, input), carrying run_id/firm_id/matter_id/batch_id + the trace carrier, and no fabricated user. Blake validates input against RegistrarDocumentInput; the consumer assembles that shape from the markdown body (quoted evidence) plus the firm/matter catalog so Blake reuses existing vocabulary instead of coining duplicates.
Oversize markdown is an honest degradation
The P2 read returns the markdown inline only up to 32 KiB; a larger document comes back as an S3 pointer (so a multi-MB file never loads into the consumer's bounded Lambda memory). Blake has no S3 keys, so for an oversize document the consumer sends an empty document_markdown and logs the degradation — Blake then routes the document to needs_review rather than guessing. Full large-document grounding is deliberately deferred; the read contract built in P2 is honored, not bypassed.
Observability: reuse the registry, never invent
The AgentCore handoff reuses the already-approved AGENT_INVOKE_BUILD / AGENT_INVOKE_CALL spans (same pattern the clerk worker uses for Atlas). The only two new closed-registry names are the consumer's batch + per-record spans, added with the standard "extends the ADR-124 closed registry; ratify with an amendment" note — exactly as the relay's batch span was added in P3a. Every hop logs trace_id + document_id + extraction_run_id, and each registration_status transition writes a trace-stamped document_status_events row, so the custody ledger and the X-Ray trace tell the same story.
How it was proven
Twenty DB-free unit tests across two layers: the pure contract logic (verdict→status map, the agent_invoke.v1 payload incl. the hop-5 traceparent, the agent_event_stream.v1 fold that reads intents out of turn.completed, the intent translation, the stream deadline) and the orchestration driven against fakes (queued→registering→registered with intents executed; the needs-review path; transport failure → registration_failed + re-raise for redelivery; the failed→requeue→re-drive retry; and the idempotent no-op on an already-registered redelivery). The real RLS/DB transitions are covered by the existing RegistrationService, executor, and status-event suites; the live end-to-end trace_id assertion is P5.
One-sentence recall: the registrar broker-consumer is the keyed dispatcher that claims a queued document, hands keyless Blake a markdown-and-catalog dossier with the trace carrier stapled on, routes Blake's proposals through the one office allowed to write truth (the broker executor), and stamps the document registered/needs_review/failed — closing the last near-side break so one trace_id runs unbroken from upload into Blake and back.