Sixteen rows, sixteen ticks.
A lawyer hands PRAMAAN a bundle: one petition with fifteen annexures stapled behind it. The knife (Story-3294) cuts it into sixteen real documents. The fan-out (Story-3295) sends sixteen clerks to read them. Each clerk takes its own time.
Until this Story, what counsel saw during that was: nothing. Sixteen files appear in a list with names derived from the carve — "Annexure A-3.pdf" — and then, minutes later and one at a time, some of them quietly get better names. There was no page that said what is in this bundle, no page that said how far along are you, and no artifact at the end that said here is the index of what you filed.
ADR-180 D19: every bundle gets an index — provisional at carve,
final from the cards. A draft appears the moment carve lands;
each row upgrades to the proper legal title as its card lands; the final
index is rendered as a PDF and filed into the family as a derived
document (provenance generated; re-made on revert or
re-carve). No model writes it. It doubles as the progress surface.
The whole thing is a projection
The index has no table of its own, no queue, no status store, and no new abstraction. It could not have any of those: it is a statement about facts three other modules already committed, and a second copy of those facts is a second thing to get out of sync.
| What the index shows | Where it already lives | Who wrote it |
|---|---|---|
| Which members, in what order, over what pages | documents.documents (family_id, annexure_position, parent_page_start/end) | materialize.py, at carve |
| The proper legal title of each | context.document_cards.production_card -> 'name' | ingest_production_card, per card drop |
| Which members have been read | agents.agent_step_drop | consume.py, when it applied the drop |
src/clerk_worker/bundle/index.py reads those three, assembles
rows, hashes them, and — if the hash moved — draws a PDF and files it.
That is the entire feature.
Seven things that are easy to get wrong, and were
1 · The generation comes from the members, not from the drop
A drop carries a generation. Building the index at the
arriving drop's generation means a straggler from a superseded carve
renders an index of members that are no longer live. So the generation is
read back off the members' own provenance manifest:
SELECT max((family_manifest->>'generation')::int)
FROM documents.documents
WHERE family_id = :bundle AND family_role = 'annexure'
AND availability = 'live' AND family_manifest->>'schema' = 'bundle_cut.v1'
Committed carve state, never queue order — and it makes "a stale generation must never become the final artifact" structural rather than a check, because a straggler recomputes the CURRENT generation and finds its members not all read.
2 · The card does not write display_name
The obvious implementation reads documents.display_name and
lets apply_record_name upgrade it as each card lands. It
would have shipped an index that sat at its provisional titles through a
fully-read bundle.
registrar.yaml ships apply_record_name: suggest.
A card's name is recorded as a PROPOSAL; nothing on this rail writes the
column. The evidence was already in the repo —
test_bundle_fanout hand-sets display_name before
asserting the D18 block, precisely because the card does not — and it was
found by watching the integration test fail with the provisional title
still on the page.
So the index reads the card directly, with counsel's approved
display_name still outranking it wherever it exists. That the
index shows a title counsel has not approved is correct: the index is a
read surface over what the machine found, not the document's name of
record.
2a · …and reading it directly is wrong without a generation gate
Found in review. context.document_cards holds ONE live card
per document and is not generation-scoped; the status table is. The main
member's document id IS the bundle's, and the knife keys an annexure's id
on its page range — so a re-carve preserves ids, and generation 1's card
is still sitting in that table when generation 2's draft is assembled.
Ungated, that draft opens with the previous generation's legal title on a
row it prints as pending, and does it again on a row whose
own card was refused. A wrong title is the worst failure this
surface has available: it is what counsel cites from, and it looks exactly
like a right one. So a card title is passed only where the
generation-scoped outcome says APPLIED; every other state falls back to
what the carve itself knew.
3 · Two final cards race, and neither is last
_settle serialises passes per DOCUMENT. Two annexures' final
cards are two different documents. Under READ COMMITTED each pass reads a
sibling set in which the other's card has not committed, each concludes
"not complete", and the bundle never gets its final index.
The two passes collide on no row, so ON CONFLICT has nothing
to arbitrate and FOR UPDATE has nothing to lock. A
transaction-scoped advisory lock on the BUNDLE is the honest tool — keyed
on the bundle alone, never on (bundle, generation), because a
lock key containing the value being compared lets two generations take
different locks and miss each other (the Story-3290 finding, one rail
over). Lock order is fixed and acyclic: document lock first, bundle lock
second, nothing takes a document lock while holding a bundle lock.
4 · A refused card must not hang the bundle forever
"Last card" cannot mean "every card APPLIED": one member whose card fails
permanently would mean a bundle that never completes and an index that is
never filed at all. Completion is every live member has a SETTLED card
row — applied, recorded_only or
refused. out_of_order_deferred is deliberately
not settled; that drop is waiting, not finished. A refused member's row
renders unread and says so, which is R5's honest terminal
state on a progress surface.
4a · …and the refusal path returns before the refresh
Found in review, and it is the same bug one level down. A drop whose
object carries forbidden custom metadata verifies its BODY and fails its
RAIL: _settle claims it, stamps the row refused,
and returns at that branch — ahead of the apply, the drain, and the index
refresh that sits after them.
So the outcome that turns a member unread, and that on the
last member turns the whole index FINAL, was the one outcome the index
never saw. Nothing later is guaranteed to fix it: the refusal burns the
drop_id, so a redelivery of the same bytes never reaches an
applier, and the filed PDF stays a draft with a pending row until a
re-carve. The refresh belongs on every terminal path, not only the
applying one.
4b · The PUT precedes the commit, so the KEY has to move
Found in review. Every refresh of an existing index wrote its bytes over
the key the already-committed document row was handing out — before the
card, the status row and the new manifest committed. Two failures at once:
a download in flight reads a transaction's uncommitted contents, and if
the database work then rolls back, the surviving row's
checksum_sha256 and manifest describe bytes that are no
longer at that key.
The key is now the digest of the bytes it holds, so every render is a NEW
object and the live one keeps its live bytes. What switches is
object_key, inside the caller's own transaction — it commits
or rolls back with everything else. Superseded renders are left in place:
they are rebuildable_derived, and a delete-after-commit has
no transaction to sit in, which is the same ordering bug pointed the other
way.
Filed into the family, and taken by a revert for free
The index document is uuid5(namespace, "{bundle}:{generation}")
— derived, so the mint is idempotent with no "does it exist yet?" read
spanning the gap; generation-keyed, so a re-carve is a new statement and
the old one is retired by a sweep scoped to rows this module minted.
It is a CHILD of the bundle (parent_document_id) with no page
range. That link is what files it into the family — and what makes
DocumentService.undo_collation take it with the members it
describes, since a revert soft-deletes every live child. There is no
revert-aware code in the index module at all. What there IS is an
availability check paired with the digest guard: a
soft-deleted index whose rows hash identically to the retired one must be
re-FILED, not skipped, or the family keeps no index forever.
It is deliberately NOT family_role='annexure', and it is
excluded from annexure_register by
retention_class = 'rebuildable_derived' (ADR-126's own
durability split). A generated index is not a tendered exhibit, and
listing it beside real annexures would put a machine artifact into
counsel's filing index. Its bytes live under derived/, not
raw/, for the same reason: it is regenerated every time a row
moves.
Zero model calls, proved two ways
A grep of index.py proves nothing on its own — Story-2169's
model call lived three imports away from the Lambda that executed it and
every CI stayed green. So the check is the repo's own R11 transitive
import walker, run from this module as its entry point, plus the literal
grep beside it. Planting an OpenRouterAdapter import reddens
both.
What a planted revert found that the tests did not
Deleting "state" from the hashed digest payload changed no
digest and reddened no test. It was redundant — the state is a pure
function of the row states — so it came out. A test whose docstring
claimed the state was what separated a draft from its own final was
claiming something true for a different reason, which is the same defect
one level up.
A second plant mattered more. Deciding completion from "every member has a
card ROW in context.document_cards" — plausible, also
committed state — passed the entire integration suite. It fails only where
a card was REFUSED: settled in the status table, no card row, so a
card-count reader waits forever. Four unit tests caught it. The
integration suite alone would have shipped it.
Each of the three review fixes above ships the same way. Reverting the
generation gate reddens
test_a_card_title_shows_only_where_THIS_generations_card_applied
and the re-carve integration proof; deleting the refresh from the
rail-refusal branch reddens
test_a_terminal_rail_refusal_still_moves_the_index; putting
the generation-only object key back reddens
test_an_update_never_overwrites_the_key_the_committed_row_exposes.
All three were run red before they were run green, and each names only
its own subject — no other test in the file moved.
The rendered artifact
Index — Writ Petition bundle
final · generation 1 · 17 of 17 read · page 1 of 1
0 Main document Petition under Section 9 of the Arbitration … pp. 1-11 read
1 Annexure A Statement of account, 2024-25 pp. 12-16 read
2 Annexure B Board resolution dated 14.03.2024 pp. 17-19 read
…
16 Annexure P Correspondence bundle pp. 27-31 unread
Drawn with pypdfium2 — already in the lockfile, already the
knife's own library — through a text-object helper extracted out of the
fetch-link watermarker so there is one copy of the ctypes wide-string
dance rather than two. A4, paginated (the knife's ceiling is 200 members;
an index that stopped at the bottom of page one would drop most of a large
bundle without erroring). No new dependency, no LaTeX, no headless
browser.
Where it sits in the pipeline
… 6 apply the ready drop, in step order
6' drain whatever it unblocked
6'' refresh the living index ← this Story, INSIDE the transaction
─── commit ───
7 fan out this bundle's clerks
8 one progress frame
Inside the transaction, so the card that upgraded a row and the index that
shows it commit together — there is no window where a member has been read
and the family's index disagrees. And deliberately NOT gated on the
arriving drop's step: a card that arrived before its classify is applied
inside the drain, so the pass whose OWN step is classify is
the pass that made that card live. The digest guard is what keeps the
drops that move nothing cheap.
The ceiling, stated
A render or upload failure is counted (outcome=render_failed)
and named, never raised. The index is a derived progress surface and the
drop that woke it carries real document truth — burning a member's card
apply because a PDF would not draw is the wrong trade, and the next drop
of the same bundle re-derives all of it. A database failure is not caught:
it has already poisoned the caller's transaction.
This pass has no manual span of its own. The closed ADR-124 registry has
no ratified name for it and inventing one fails
span-registry-gate.py; it runs inside
functions.step_drop.consume, so the work is on the trace
already. What it adds is the counter — registered in pramaan-docs first,
because the metric-registry gate reads that table from main
and a local METRIC_SPECS entry is necessary and not
sufficient.