Story-3294 · the P3 knife · pramaan-functions

An annexure stops being a page range

A lawyer who wants Annexure A-2 wants a PDF. They want to open it, mail it, attach it to the next application, and see on its face which pages of which bundle it came from. Until this Story a carve member was a page RANGE — a row that pointed into its parent and owned nothing. ADR-180 D17 settles it: the broker holds the PDF and the knife. On the carve event it cuts each member into a real PDF, registers it as a document with its own id and provenance, and ties main plus annexures into one family. The sealed original is never edited.

P2 landed the rail that gets a carve drop here. This Story is the applier it dies in: apply_carve stopped saying carve_materialization_pending and started cutting.

The three ways a knife is silently wrong

Every decision below exists because of one of these, and none of them raises anything on its own.

FailureWhat it looks likeWhat stops it
Off by one page A perfectly well-formed PDF of the WRONG pages. Every downstream check passes; the citation points at a page that was never in that annexure. The carve family is 1-based and PDFium is 0-based, so the conversion lives in exactly one place. The proof gives every page of a synthetic bundle a DIFFERENT page size, so "this cut carries pages 3–4" is an assertion about bytes rather than about the arithmetic that made them.
A second row per redelivery S3 → EventBridge → SQS is at-least-once. Mint a fresh uuid4 per pass and one bundle quietly becomes thirty-two annexures in the matter. Identity is CONTENT-DERIVED: uuid5(bundle_document_id + page range). Same bundle, same seam, same document — forever. The mint is INSERT … ON CONFLICT (id) DO NOTHING RETURNING id, so the database arbitrates and a lost claim is an expected outcome, never an exception (root AGENTS.md R14). Losing the claim also SKIPS the cut, so a re-drive costs zero PUTs.
Half a bundle Ten of sixteen annexures in the matter looks exactly like sixteen to anyone who was not counting. Nobody comes back for the other six. Two measured ceilings, and over either one the pass refuses WHOLE with a named outcome instead of cutting what fits.

The ceilings, with their numbers

The knife itself is free. Measured on pypdfium2 5.10.1 — the same wheel the Lambda runs — a 75 MB / 120-page bundle cut into 120 one-page members took 33 ms end to end (0.3 ms/member), and into 18 seven-page members 31 ms (1.7 ms/member). A real 8.3 MB / 11-page bundle cut 11 ways took 9 ms. What costs is I/O: one GET of the source, then one PUT + one INSERT + one UPDATE per member.

CeilingNumberWhere it comes from
MEMBER_CEILING 200 members At a conservative 1 s per member (PUT round trip through the VPC S3 gateway endpoint plus two statements) that is 200 s inside the clerk-worker's 900 s function timeout, against an SQS visibility of 1320 s — leaving >600 s for the source GET, the ordered drain and the progress push. Comfortably past the ticket's "100+ member bundles".
SOURCE_BYTES_CEILING 250 MB Peak RSS measured at 205 MB holding a 75 MB source with PDFium loaded — about 2.7× the source. The worker has 1024 MB, so 250 MB needs ~675 MB and leaves ~350 MB for the runtime, asyncpg and the largest single member's output. Infra allows 300–500 MB uploads, so this IS reachable: refusing with a name beats an OOM kill, which leaves no terminal state at all.

Chunked self-continuation is the stated upgrade path for the first bundle that exceeds 200 members, and it is already cheap: because identity is content-derived, a second pass mints only what is missing. It is not built because nothing has needed it, and a resumption mechanism nobody has exercised is a second thing that can be silently wrong.

Real bytes, borrowed text

A member owns bytes but NOT an extraction. ADR-180 §1 is explicit — "there is no re-OCR; a member's text is a page-range slice of the bundle's already-extracted text" — so the member row carries family_source_extraction_version_id plus its parent_page_start/end and its boundary anchors, which is exactly the pointer the shipped read path already resolves. This Story writes zero extraction_versions rows and zero page rows.

That is also why the coexistence window costs nothing. A virtual child minted by the pre-knife path (Story-2512) answers the SAME pointer question; only the bytes differ. Both shapes resolve through one branch of _document_source, which is what keeps existing children readable until the P5 cutover deletes the old minting path. 

cut member                       virtual child (pre-knife)
─────────────────────────        ─────────────────────────────
object_key  → real PDF           object_key  → nothing ever PUT
size_bytes  → len(cut)           size_bytes  → NULL
checksum    → sha256(cut)        checksum    → NULL
parent_page_start/end  ────────  parent_page_start/end
family_source_extraction_version_id  ──  same column, same read path

The pass

apply_carve(session, ctx, drop)
  │
  ├─ parse_carve_family              BundleFamily — validated BEFORE the claim,
  │                                  so a malformed family cannot burn a drop_id
  └─ materialize_bundle_members      commit-gated span, outcome pessimistic
       │
       ├─ family names this document?          else refused_family_identity
       ├─ compound, with members?              else carve_found_no_members
       ├─ ≤ MEMBER_CEILING?                    else refused_member_ceiling
       ├─ ranges contiguous & exhaustive?      else refused_family_ranges
       ├─ bundle row + CURRENT extraction      else refused_no_current_extraction
       ├─ family's extraction still current?   else refused_stale_extraction_version
       ├─ source is a PDF?                     else refused_source_not_pdf
       ├─ every boundary page has anchors?     else refused_member_range_has_no_anchors
       ├─ GET the sealed original ONCE         AccessDenied ⇒ refused_source_unreadable
       │                                       (permanent — it does NOT retry to the DLQ)
       ├─ ≤ SOURCE_BYTES_CEILING?              else refused_source_bytes_ceiling
       ├─ sha256 == the row's digest?          else refused_source_digest_mismatch
       ├─ cut every member                     range outside the PDF ⇒ refused_page_count_mismatch
       ├─ sha256 unchanged after the cuts      else refused_source_digest_mismatch
       │
       └─ per member: claim (ON CONFLICT DO UPDATE) → cut → PUT → status event
            ├─ then: retire the members this family no longer names
            └─ then: bundle becomes main_document of its own family

Fourteen exits, fourteen names. The span's outcome, the status row's reason and the census log line all carry the same word, and the span is commit-gated — it ends when the caller's transaction commits, so a rollback exports the pessimistic value rather than a clean trace over work that was erased (root AGENTS.md R12).

Three status words, not two

Review caught the knife collapsing every non-success onto recorded_only. It is the wrong word for thirteen of the fourteen exits, and wrong in the dangerous direction: the drop_id is already claimed, so no redelivery will ever revisit that carve, and the status row is all that is left of it. A permanently failed bundle sat in the table wearing the same harmless label as a healthy single document that simply had nothing to cut.

StatusMeans
appliedMembers were cut and registered.
recorded_only carve_found_no_members and nothing else — a single document misread as a bundle. A successful no-op.
refused Any of the thirteen refusals. A terminal FAILURE: no applied fact rides the outbox for it, and the recovery is a re-carve at the next generation once the named cause is fixed.

What a re-carve leaves behind

A range-derived id has a corollary that is easy to miss: a re-carve that MOVES a seam does not update anything, it mints a different document. The old rows stay live children of the same parent, and both list_documents and annexure_register keep returning them — so counsel sees annexures from two generations at once, both looking equally current, while the parent advertises only the newest family. Review caught that too.

So the pass retires what the current family no longer names: soft-delete, never session.delete(), and a member under legal hold is left alone and reported rather than failing the whole carve. A member that a KEPT seam merely renames is reconciled instead — the conflict is DO UPDATE, not DO NOTHING, so the label, the filename, the provenance generation, the extraction pointer and the boundary anchors all move to the new generation while the bytes (which cannot have changed, since the page range is what the id is derived from) are neither re-cut nor re-PUT.

The retirement is scoped to rows THIS RAIL minted, by the provenance schema on family_manifest — and that scoping is not a nicety. A pre-knife virtual child is also a live annexure of the same family, at a range the carve family does not name. The first version retired those too, which would have closed the coexistence window the moment anyone re-carved. The coexistence proof went red on exactly that. Retiring them is P5's decision, with a migration behind it, not a side effect of a carve.

The sealed original

Two guards, because "we never write to it" is a claim and not a mechanism. _put_member_object refuses outright to address the source key — structurally impossible, since the member key embeds the member's uuid5, which is precisely why it is an assertion: if it ever fires, a refactor has taught the knife to overwrite a WORM original with one of its own annexures, and the document list would look entirely normal. And every carve hashes the source before the first cut and after the last.

What is not here

Where the seams are