Sixteen documents, sixteen clerks, and one of them goes first
Story-3294 gave a carved bundle real member PDFs. Nothing read them. A 307-page filing became one main document and fifteen annexures that existed as files, carried their provenance, resolved their page slices — and had no name, no card, no sections, because no clerk had ever looked at one. This Story is the part that looks: every member gets its own clerk run, and the main member's runs first.
Why "first" is the whole ticket
An annexure read cold is "a two-page bank statement". Read beside the filing it belongs to, it is "the statement the petitioner relies on for ground III". ADR-180 D18 is that sentence turned into a rail: the annexure clerks read the main member's classify and card as context, so those two have to exist before the annexure clerks start.
There is an easy wrong way to do this and it is a timer. Wait thirty seconds, then fan out. It works on the first bundle, it is wrong the first time a model call is slow, and it is wrong silently — the annexures still get clerked, they just get clerked against a context that was not there, and nothing anywhere goes red. So the barrier here is not a wait. It is a read:
async def main_member_is_read(session, *, firm_id, bundle_document_id, generation) -> bool:
applied = await drop_status.applied_steps_for_document(
session, firm_id=firm_id, document_id=bundle_document_id, generation=generation
)
return {"classify", "card"} <= applied
Those rows are in agents.agent_step_drop. They were written by
the transaction that applied those drops, and they were committed by it.
The fan-out opens its own session after the consumer's transaction
has committed and reads them back, rather than carrying a conclusion out of
the transaction that staged it — a barrier proved from a variable in this
process is a barrier that a rollback makes wrong while the trace still
looks clean.
Where the main member actually lives
Member 0 is the bundle row itself. pramaan-agents'
carve_drop.MAIN_MEMBER = 0 numbers the head as member 0 and
the annexures 1..N. But the knife cuts family.members only —
it never cuts the head — and it stamps the bundle document
family_role='main_document', which is the mapping collation
has always used. So "invoke the main member's clerk" means: invoke a
clerk on the bundle document.
That one fact decided the claim. The obvious claim to reuse was
claim_registration — "has this document been handed to the
registrar?" — and for an annexure it would have been exactly right. For the
main member it would have been a permanent no-op: the bundle's registration
claim is already held by the registration pass that detected the
bundle in the first place, so main-first would have silently invoked
nothing, forever, on every bundle. That is the EPIC's own named trap wearing
a claim table for a costume.
So migration 0246 adds a clerk_fanout boundary to
agents.agent_idempotency_key — the same table, one boundary
wider, mirroring 0245's carve trigger line for line. The digest is
(bundle_document_id, member_document_id, generation).
The order, and why every step of it is load-bearing
- The capability, before anything durable. A claim taken against a signing capability that cannot mint is a claim held forever by a clerk that never ran, recoverable only by hand. Story-3289 paid for this lesson; this reuses the ordering rather than re-deriving it.
-
The claim and the pending
agents.agent_runrow, in one transaction that commits. A death between commit and invoke leaves a durable, sweeper-visible row rather than a clerk nobody recorded. -
Mint and dispatch, in parallel. Fifteen annexure clerks
have nothing to say to each other; serialising fifteen ~2 s broker round
trips onto one SQS message's clock is exactly the latency ADR-180 exists
to delete.
return_exceptions=True, because one dead runtime must not cancel fourteen healthy invocations whose claims are already committed.
The trigger is not "which drop am I"
The obvious wiring is: fan out when the drop I just applied is the main
member's card. It is wrong, and the reason is a feature of the rail rather
than a bug. A card that arrives before its classify is DEFERRED, and it is
applied later inside _drain_deferred — so the pass whose
primary drop is the classify is the pass that made the card live.
Gate on drop.step == "card" and that bundle never fans out.
So every settled drop for a split bundle re-reads the committed truth and re-offers every member. The per-member claim makes the repeated offer free, makes the first offer that reaches an unclaimed member the only one that invokes, and makes the whole thing self-healing: a member whose claim never committed is picked up by the next drop of the same bundle.
It also means the replay proof is a real proof. A redelivered card settles
duplicate_claimed, and the fan-out evaluates on that outcome
deliberately — so the replay reaches the claim and is refused by
the claim, rather than being short-circuited by an early return that would
make the test pass without a claim existing at all.
What rides the annexure envelopes
D18 is on the wire. P4 is merged
(pramaan-agents#422, Story-3296), so every annexure clerk receives
journey_context: the family pointer, the MAIN member's
classify and card reads, and the index of every member of the bundle.
Main-first and that block are one mechanism, not two — the barrier is
precisely what guarantees the block is non-empty when it is built.
An intermediate cut of this Story shipped the ordering with the annexure dispatch live and nothing on the wire. That is wrong in the way that is hardest to see: fifteen clerks that classify, card and segment a page range with no idea which filing it belongs to — and persist all of it. Ordering with nothing riding on it is not a smaller version of the feature. It is the feature's cost with none of its benefit. The answer is to put the benefit on the wire, not to keep the cost.
The guard stays, and it is not belt-and-braces. It asks this module's own
envelope builder whether the block reaches an annexure's input:
def annexure_dispatch_is_live() -> bool:
probe = build_clerk_invoke_payload(..., journey_context=_PROBE_JOURNEY)
return D18_JOURNEY_KEY in probe["input"]
Never a flag, so a flag and the wire can never disagree. If a refactor ever
drops the block, the annexures stop rather than run cold — the safe
direction — and the held branch counts itself on
pramaan.bundle.clerk_fanout_skipped instead of passing in
silence.
The main member gets no block, for two independent reasons
The far side renders an empty block for a non-annexure anyway, so handing a
document its own card back is a self-referential prompt and 2,000 wasted
characters. And it would not survive the key grammar. The main member's
ordinal is 0; a journey_context naming it makes the brain stamp
member=0 onto its drops, and the drop-key grammar cannot express
member 0 at all — measured, not assumed:
compose_step_drop_key(..., member=None) -> runs/<run>/steps/1-classify.json
compose_step_drop_key(..., member=1) -> runs/<run>/steps/1-classify.member-1.json
compose_step_drop_key(..., member=0) -> REFUSED: outside the broker-minted grammar
The key would carry no member, the body would carry 0, and every one of the
main member's three drops would be refused
refused_key_body_mismatch. That producer/consumer numbering
mismatch is pramaan-spec Bug-3308; this is the shape that
works around it without pretending it is not there. The corollary is that
an annexure's three step envelopes must be minted at
…member-<k>.json, because the brain stamps that same
ordinal onto its drops from the journey block and
assert_key_matches_body compares the two.
Where the block's content comes from
All committed truth, read once per fan-out: display_name and
the drawer label off the bundle row (classify's writes), and
doc_type, summary and the party names off
context.document_cards.production_card (the card's). Not the
card column — that is the legacy DocumentCard
writer, whose summary is a bare string rather than a cited
value, and reading it returns NULL for every field and a D18 block that is
all page ranges. Measured, not assumed.
A permission gap must not eat a claim
Review's second finding: build_artifact_delivery_deps
constructs clients and reads config — it makes no AWS call — and
resolve_runtime_arn proves SSM and nothing else. So "the
capability was built" proved nothing about the grant, and the first real
sts:AssumeRole happened inside the mint, one statement
after the claims were committed. A role without that grant burned
every member of the bundle, and a claim keyed on a substrate rather than a
run is unrecoverable without a re-carve.
probe_signing_capability now assumes the role before the
claim, through the same cache the mint uses — one STS call on a cold
container, zero on a warm one. The invoke grant has no pre-check to add: the
clerk is a mode on record_registrar, the same runtime under the
same enumerated grant the carve trigger uses, and this code only ever runs
in response to a carve drop that exists because that invoke already
succeeded. Building a release path for it would have needed a
DELETE grant on agents.agent_idempotency_key the
app role does not hold — widening its authority over every registration
claim in the firm to guard a case that cannot arise.
And the loser's run key goes with it
Each offer mints a fresh agents.agent_run_key before it knows
whether its claim will win, because the claim is FK-chained to it. Since the
fan-out re-offers every member on every drop of the bundle by design, a
losing claim that kept its key leaked one durable orphan per already-claimed
member per pass. It is deleted on the losing path — the uuid4 was created in
that same transaction and nothing else can reference it, so the delete is
exact rather than a sweep.
The envelope
mode="clerk" — one member, three steps, three drops. The three
per-step inputs are the shipped per-pass builders' own input
blocks, not a fourth spelling of them: classify flat (the registration
invoke shape), card and segment nested, which is the shape pramaan-agents'
runtime._clerk_step_input reads. The nesting is load-bearing —
the card step needs an anchored page spine the flat registration envelope
does not carry, and a card step that fails drops no card, and a card that
never drops is a bundle that never fans out.
A member owns bytes but no extraction, and none of that needed new
plumbing: list_card_pages follows
family_source_extraction_version_id back to the bundle's
extraction and slices it to the member's page range, and
read_document_model_view materialises the same slice. Both are
the shipped resolvers, doing what Story-3294's docstring promised they
would.
The span
functions.bundle.fanout was ratified in the ADR-124 registry
with zero emission sites. This is the ticket that fans out, so it emits it:
through the shared manual_span helper, at the ratified name,
SpanKind.PRODUCER, and the allowlisted attributes
{firm_id, run_id, outcome} and nothing else. It opens
pessimistically at fanout_incomplete and is overwritten only by
a terminal word.
outcome | what happened |
|---|---|
main_invoked | a clerk was dispatched and the main member was among them |
annexures_invoked | clerks were dispatched and all of them were annexures |
awaiting_main | the barrier is holding — the main member is claimed, its card has not landed |
already_invoked | every member was already claimed. The healthy repeat |
no_members | a split bundle the knife cut nothing out of |
no_brain_seams / signing_capability_absent | nothing to invoke, or nothing that can mint an envelope |
invoke_failed | at least one dispatch raised after its claim committed |
A bundle stalled behind its main member and a bundle that fanned out must never look the same in the trace — that is why the vocabulary is a closed set of words and not a boolean. The span is deliberately NOT opened for a drop whose document is not a split bundle: that is the overwhelming majority of every clerk-worker message, and opening it on each of them would make the one span that means "a bundle fanned out" mean nothing at all.
Both proofs were watched fail
No green counts until it has been red. Two reverts were planted into
fanout.py, run, and reverted:
| planted revert | what went red |
|---|---|
the barrier deleted — targets = [main] + annexures, unconditionally |
4 failed, 6 passed |
the per-member claim never refuses — claimed = True |
6 failed, 4 passed |
| the D18 gate deleted — annexures dispatch without their context | 2 failed, 36 passed |
the signing probe moved back after _claim_and_prepare |
1 failed, 37 passed |
| the losing claim keeps its run key | 1 failed, 37 passed |
| the D18 block stops reaching the annexure envelope | 9 failed, 33 passed |
| the step envelopes stop naming the member | 1 failed, 41 passed |
| the main member is handed its own card (member 0 on the wire) | 16 failed, 26 passed |
The first plant found a weak test, which is the point of planting it. The barrier proof originally cleared the recorded invocations between the carve drop and the classify drop — and with the barrier deleted it stayed GREEN, because by the classify drop every annexure the carve pass wrongly invoked was already claimed and the second pass looked quiet. The assertion is now cumulative across both drops, and the same plant reddens it.
The ceiling, stated
An invoke that fails after its claim committed is not retried by this
module: the claim is held and the drop that triggered it is already
consumed. It is not lost — write_pending_artifact_run left a
durable agents.agent_run row with
delivery_status='awaiting_artifact' and a deadline, which is
the sweeper's input, and the broker censuses
invoke_failed_pending_row_visible on the way out. The recovery
for a whole bad bundle stays ADR-180's own: revert and re-carve at
generation + 1, which hashes to fresh claims for every member.
What still blocks the staging proof
The merged carver's drop payload does not validate as the merged
knife's BundleFamily. pramaan-agents
carve_drop.build_carve_step_drop writes
{status, bundle_id, schemes, candidate_count, members:[…]};
apply_carve parses drop.payload as
BundleFamily, which requires
source_document_id, page_count and
head, and forbids the member/role
keys the drop's member rows carry. Measured, not inferred: nine
validation errors. Every real carve drop therefore settles
refused_payload and the knife never cuts, so nothing
downstream of it — including this fan-out — can run on staging until one
side moves. It is Story-3294's contract with agents#417, not this
Story's, and it is raised rather than silently patched here.