Story-3295 · the P3 fan-out · pramaan-functions

Sixteen documents, sixteen clerks, and one of them goes first

Story-3294 gave a carved bundle real member PDFs. Nothing read them. A 307-page filing became one main document and fifteen annexures that existed as files, carried their provenance, resolved their page slices — and had no name, no card, no sections, because no clerk had ever looked at one. This Story is the part that looks: every member gets its own clerk run, and the main member's runs first.

Why "first" is the whole ticket

An annexure read cold is "a two-page bank statement". Read beside the filing it belongs to, it is "the statement the petitioner relies on for ground III". ADR-180 D18 is that sentence turned into a rail: the annexure clerks read the main member's classify and card as context, so those two have to exist before the annexure clerks start.

There is an easy wrong way to do this and it is a timer. Wait thirty seconds, then fan out. It works on the first bundle, it is wrong the first time a model call is slow, and it is wrong silently — the annexures still get clerked, they just get clerked against a context that was not there, and nothing anywhere goes red. So the barrier here is not a wait. It is a read:

async def main_member_is_read(session, *, firm_id, bundle_document_id, generation) -> bool:
    applied = await drop_status.applied_steps_for_document(
        session, firm_id=firm_id, document_id=bundle_document_id, generation=generation
    )
    return {"classify", "card"} <= applied

Those rows are in agents.agent_step_drop. They were written by the transaction that applied those drops, and they were committed by it. The fan-out opens its own session after the consumer's transaction has committed and reads them back, rather than carrying a conclusion out of the transaction that staged it — a barrier proved from a variable in this process is a barrier that a rollback makes wrong while the trace still looks clean.

Where the main member actually lives

Member 0 is the bundle row itself. pramaan-agents' carve_drop.MAIN_MEMBER = 0 numbers the head as member 0 and the annexures 1..N. But the knife cuts family.members only — it never cuts the head — and it stamps the bundle document family_role='main_document', which is the mapping collation has always used. So "invoke the main member's clerk" means: invoke a clerk on the bundle document.

That one fact decided the claim. The obvious claim to reuse was claim_registration — "has this document been handed to the registrar?" — and for an annexure it would have been exactly right. For the main member it would have been a permanent no-op: the bundle's registration claim is already held by the registration pass that detected the bundle in the first place, so main-first would have silently invoked nothing, forever, on every bundle. That is the EPIC's own named trap wearing a claim table for a costume.

So migration 0246 adds a clerk_fanout boundary to agents.agent_idempotency_key — the same table, one boundary wider, mirroring 0245's carve trigger line for line. The digest is (bundle_document_id, member_document_id, generation).

The order, and why every step of it is load-bearing

  1. The capability, before anything durable. A claim taken against a signing capability that cannot mint is a claim held forever by a clerk that never ran, recoverable only by hand. Story-3289 paid for this lesson; this reuses the ordering rather than re-deriving it.
  2. The claim and the pending agents.agent_run row, in one transaction that commits. A death between commit and invoke leaves a durable, sweeper-visible row rather than a clerk nobody recorded.
  3. Mint and dispatch, in parallel. Fifteen annexure clerks have nothing to say to each other; serialising fifteen ~2 s broker round trips onto one SQS message's clock is exactly the latency ADR-180 exists to delete. return_exceptions=True, because one dead runtime must not cancel fourteen healthy invocations whose claims are already committed.

The trigger is not "which drop am I"

The obvious wiring is: fan out when the drop I just applied is the main member's card. It is wrong, and the reason is a feature of the rail rather than a bug. A card that arrives before its classify is DEFERRED, and it is applied later inside _drain_deferred — so the pass whose primary drop is the classify is the pass that made the card live. Gate on drop.step == "card" and that bundle never fans out.

So every settled drop for a split bundle re-reads the committed truth and re-offers every member. The per-member claim makes the repeated offer free, makes the first offer that reaches an unclaimed member the only one that invokes, and makes the whole thing self-healing: a member whose claim never committed is picked up by the next drop of the same bundle.

It also means the replay proof is a real proof. A redelivered card settles duplicate_claimed, and the fan-out evaluates on that outcome deliberately — so the replay reaches the claim and is refused by the claim, rather than being short-circuited by an early return that would make the test pass without a claim existing at all.

What rides the annexure envelopes

D18 is on the wire. P4 is merged (pramaan-agents#422, Story-3296), so every annexure clerk receives journey_context: the family pointer, the MAIN member's classify and card reads, and the index of every member of the bundle. Main-first and that block are one mechanism, not two — the barrier is precisely what guarantees the block is non-empty when it is built.

An intermediate cut of this Story shipped the ordering with the annexure dispatch live and nothing on the wire. That is wrong in the way that is hardest to see: fifteen clerks that classify, card and segment a page range with no idea which filing it belongs to — and persist all of it. Ordering with nothing riding on it is not a smaller version of the feature. It is the feature's cost with none of its benefit. The answer is to put the benefit on the wire, not to keep the cost.

The guard stays, and it is not belt-and-braces. It asks this module's own envelope builder whether the block reaches an annexure's input:

def annexure_dispatch_is_live() -> bool:
    probe = build_clerk_invoke_payload(..., journey_context=_PROBE_JOURNEY)
    return D18_JOURNEY_KEY in probe["input"]

Never a flag, so a flag and the wire can never disagree. If a refactor ever drops the block, the annexures stop rather than run cold — the safe direction — and the held branch counts itself on pramaan.bundle.clerk_fanout_skipped instead of passing in silence.

The main member gets no block, for two independent reasons

The far side renders an empty block for a non-annexure anyway, so handing a document its own card back is a self-referential prompt and 2,000 wasted characters. And it would not survive the key grammar. The main member's ordinal is 0; a journey_context naming it makes the brain stamp member=0 onto its drops, and the drop-key grammar cannot express member 0 at all — measured, not assumed:

compose_step_drop_key(..., member=None) -> runs/<run>/steps/1-classify.json
compose_step_drop_key(..., member=1)    -> runs/<run>/steps/1-classify.member-1.json
compose_step_drop_key(..., member=0)    -> REFUSED: outside the broker-minted grammar

The key would carry no member, the body would carry 0, and every one of the main member's three drops would be refused refused_key_body_mismatch. That producer/consumer numbering mismatch is pramaan-spec Bug-3308; this is the shape that works around it without pretending it is not there. The corollary is that an annexure's three step envelopes must be minted at …member-<k>.json, because the brain stamps that same ordinal onto its drops from the journey block and assert_key_matches_body compares the two.

Where the block's content comes from

All committed truth, read once per fan-out: display_name and the drawer label off the bundle row (classify's writes), and doc_type, summary and the party names off context.document_cards.production_card (the card's). Not the card column — that is the legacy DocumentCard writer, whose summary is a bare string rather than a cited value, and reading it returns NULL for every field and a D18 block that is all page ranges. Measured, not assumed.

A permission gap must not eat a claim

Review's second finding: build_artifact_delivery_deps constructs clients and reads config — it makes no AWS call — and resolve_runtime_arn proves SSM and nothing else. So "the capability was built" proved nothing about the grant, and the first real sts:AssumeRole happened inside the mint, one statement after the claims were committed. A role without that grant burned every member of the bundle, and a claim keyed on a substrate rather than a run is unrecoverable without a re-carve.

probe_signing_capability now assumes the role before the claim, through the same cache the mint uses — one STS call on a cold container, zero on a warm one. The invoke grant has no pre-check to add: the clerk is a mode on record_registrar, the same runtime under the same enumerated grant the carve trigger uses, and this code only ever runs in response to a carve drop that exists because that invoke already succeeded. Building a release path for it would have needed a DELETE grant on agents.agent_idempotency_key the app role does not hold — widening its authority over every registration claim in the firm to guard a case that cannot arise.

And the loser's run key goes with it

Each offer mints a fresh agents.agent_run_key before it knows whether its claim will win, because the claim is FK-chained to it. Since the fan-out re-offers every member on every drop of the bundle by design, a losing claim that kept its key leaked one durable orphan per already-claimed member per pass. It is deleted on the losing path — the uuid4 was created in that same transaction and nothing else can reference it, so the delete is exact rather than a sweep.

The envelope

mode="clerk" — one member, three steps, three drops. The three per-step inputs are the shipped per-pass builders' own input blocks, not a fourth spelling of them: classify flat (the registration invoke shape), card and segment nested, which is the shape pramaan-agents' runtime._clerk_step_input reads. The nesting is load-bearing — the card step needs an anchored page spine the flat registration envelope does not carry, and a card step that fails drops no card, and a card that never drops is a bundle that never fans out.

A member owns bytes but no extraction, and none of that needed new plumbing: list_card_pages follows family_source_extraction_version_id back to the bundle's extraction and slices it to the member's page range, and read_document_model_view materialises the same slice. Both are the shipped resolvers, doing what Story-3294's docstring promised they would.

The span

functions.bundle.fanout was ratified in the ADR-124 registry with zero emission sites. This is the ticket that fans out, so it emits it: through the shared manual_span helper, at the ratified name, SpanKind.PRODUCER, and the allowlisted attributes {firm_id, run_id, outcome} and nothing else. It opens pessimistically at fanout_incomplete and is overwritten only by a terminal word.

outcomewhat happened
main_invokeda clerk was dispatched and the main member was among them
annexures_invokedclerks were dispatched and all of them were annexures
awaiting_mainthe barrier is holding — the main member is claimed, its card has not landed
already_invokedevery member was already claimed. The healthy repeat
no_membersa split bundle the knife cut nothing out of
no_brain_seams / signing_capability_absentnothing to invoke, or nothing that can mint an envelope
invoke_failedat least one dispatch raised after its claim committed

A bundle stalled behind its main member and a bundle that fanned out must never look the same in the trace — that is why the vocabulary is a closed set of words and not a boolean. The span is deliberately NOT opened for a drop whose document is not a split bundle: that is the overwhelming majority of every clerk-worker message, and opening it on each of them would make the one span that means "a bundle fanned out" mean nothing at all.

Both proofs were watched fail

No green counts until it has been red. Two reverts were planted into fanout.py, run, and reverted:

planted revertwhat went red
the barrier deleted — targets = [main] + annexures, unconditionally 4 failed, 6 passed
the per-member claim never refuses — claimed = True 6 failed, 4 passed
the D18 gate deleted — annexures dispatch without their context 2 failed, 36 passed
the signing probe moved back after _claim_and_prepare 1 failed, 37 passed
the losing claim keeps its run key 1 failed, 37 passed
the D18 block stops reaching the annexure envelope 9 failed, 33 passed
the step envelopes stop naming the member 1 failed, 41 passed
the main member is handed its own card (member 0 on the wire) 16 failed, 26 passed

The first plant found a weak test, which is the point of planting it. The barrier proof originally cleared the recorded invocations between the carve drop and the classify drop — and with the barrier deleted it stayed GREEN, because by the classify drop every annexure the carve pass wrongly invoked was already claimed and the second pass looked quiet. The assertion is now cumulative across both drops, and the same plant reddens it.

The ceiling, stated

An invoke that fails after its claim committed is not retried by this module: the claim is held and the drop that triggered it is already consumed. It is not lost — write_pending_artifact_run left a durable agents.agent_run row with delivery_status='awaiting_artifact' and a deadline, which is the sweeper's input, and the broker censuses invoke_failed_pending_row_visible on the way out. The recovery for a whole bad bundle stays ADR-180's own: revert and re-carve at generation + 1, which hashes to fresh claims for every member.

What still blocks the staging proof

The merged carver's drop payload does not validate as the merged knife's BundleFamily. pramaan-agents carve_drop.build_carve_step_drop writes {status, bundle_id, schemes, candidate_count, members:[…]}; apply_carve parses drop.payload as BundleFamily, which requires source_document_id, page_count and head, and forbids the member/role keys the drop's member rows carry. Measured, not inferred: nine validation errors. Every real carve drop therefore settles refused_payload and the knife never cuts, so nothing downstream of it — including this fan-out — can run on staging until one side moves. It is Story-3294's contract with agents#417, not this Story's, and it is raised rather than silently patched here.