Story 1866: page → segment — teaching the citation model to say “slide 3” without lying

Why this exists (the catastrophe it prevents). A lawyer drops a real case folder: PDFs, scans, a Word brief, an Excel damages sheet, a PowerPoint timeline. Today only PDFs and images extract; everything else lands honestly unreadable — safe on the Record, but invisible to search, evidence-text, and Blake. RFD-DOC-002 fixes that by rendering Office docs to PDF and parsing spreadsheets. But the entire shipped pipeline is built on one word: page. document_pages, per-page bounding boxes, “cite page N.” A .pptx has no pages — it has slides. A .xlsx has no pages — it has sheets, rows, cells. If we force those into a page number, a citation reads “page 4” for what counsel sees as “slide 4” or “Sheet ‘Damages’, row 12” — a citation that does not match the exhibit is a citation a court can strike. This Story is the schema foundation: it generalises the unit from page to segment, mints the immutable anchor a citation binds to, and names the new failure reasons — so the seven logic stories that follow (S2–S8) build on a model that can cite every format honestly, with zero further migrations.

🛰 Satellite view: one new word, three additive moves, one migration

RFD-DOC-002’s panel locked a single load-bearing decision (DR-8): stop hard-coding “page,” generalise the unit to a segment with a kind. This Story owns all of the V2 schema in one migration (0137, single alembic head); S2–S8 are logic-only and add no migrations of their own. Three additive, all-nullable moves — nothing existing is rewritten, so every page row that exists today reads back unchanged:

✈️ Airplane view: how a citation stays correct across formats

A persisted page row used to answer one question — “which page?” The segment model makes it answer a typed pair: which kind of unit, and which one. The resolver is the single translator, and its back-compat rule is the whole trick:

  document_pages row                 resolve_segment()              renders as
  ------------------                 -----------------              ----------
  kind='page'  index=NULL  pg=4  ->  (page,  4)                ->   "page 4"      (legacy / PDF, unchanged)
  kind='slide' index=3     pg=3  ->  (slide, 3)                ->   "slide 3"     (PPTX, set in S5)
  kind='sheet' index=2  row=11
               cell='B12'        ->  (sheet, 2, row 11, B12)   ->   "Sheet 2, B12"  (XLSX, set in S5)

The rule: segment_index falls back to page_number when unset. So every row written before this Story — and every PDF written after it — resolves to exactly the page citation it always did. No backfill, no rewrite of the persist path, no behaviour change for the format that ships today. The new kinds light up only when S5 sets them.

🔬 Microscopic view: the decisions that matter

1. Why segment_index is nullable, not backfilled

The tempting move is to backfill segment_index = page_number for every existing row and make the column authoritative. It buys nothing: the OCR/native persist services don’t set segment_index, so a fresh page row would write NULL again the next day, and the resolver would still need the fallback. Backfilling is churn that doesn’t remove the fallback it was meant to remove. So the column stays nullable, the resolver owns the fallback, and the persist path is untouched — segment_kind’s server-default of 'page' is the only thing a legacy insert relies on, and the database supplies it.

2. The render anchor is immutable, and that is the point (DR-1)

An Office doc has no pages until a renderer lays it out with a font, a margin, a paper size. Change the LibreOffice version and “page 6” can become “page 7” — silently re-pointing a citation a lawyer already relied on. The defence is to make the render a thing with an identity: render_id names it, render_version pins the exact LibreOffice image digest it was produced with (never :latest), render_content_sha256 fixes its bytes, and rendered_pdf_key is the write-once artifact. A citation binds to the render_id, not to “whatever the renderer produces today.” All four are nullable because only Office docs render — a PDF or image leaves them NULL and keeps citing its own native pages. The render stage (S2) mints them; this Story only lays the columns down.

3. The reason code is coupled to its status — honesty enforced by CHECK (DR-4/DR-13)

The new terminals need a machine-readable reason the app can translate and badge, distinct from the free-text human detail that already rides document_status_events.error. So processing_status_reason carries a code — but a code without coupling is just a tag, and an evidence system cannot afford a row that claims an incoherent state. The CHECK ties each code to the one status that may carry it:

  processing_status_reason IS NULL
  OR (processing_status = 'unreadable'   AND reason = 'render_failed')
  OR (processing_status = 'needs_review' AND reason IN ('render_partial', 'ocr_budget'))

So an unreadable document can never claim it merely blew the OCR budget, and a needs_review document can never claim the render hard-failed. The vocabulary and the state-machine honesty are one constraint — an extra reason in a future story is a deliberate CHECK widen, exactly how mig 0136 widened the processing_status enum to add unreadable.

4. RLS is unchanged — and that is a claim, not an omission

Every move here is an additive column or CHECK on documents.documents and documents.document_pages. Both tables already FORCE row-level security with firm-isolation policies (migs 0031 / 0113); new columns inherit that isolation with nothing to add. There is no new table, so there is no new policy to write — the “RLS written by hand” rule has no surface to apply to here, and the migration deliberately touches none.

5. Observability: the resolver is instrumentation-exempt by the floor (pramaan-docs ADR-124)

resolve_segment is a pure in-memory function — no I/O, no DB, no boundary crossing — so it is covered by its caller’s span and adds no manual span of its own. ADR-124 reserves manual spans for boundary operations; a pure coordinate translation is exactly the “small pure helper” the guardrail exempts. No raw doc text touches telemetry: the resolver returns coordinates (kind, index, row, cell), never page content.

Verification

The full round-trip and alembic check run clean against make test-db; ruff is clean on every changed file; git diff --check is clean; openapi/app-api.json and admin-api.json are unchanged (no DTO or route touched — the segment columns are not yet surfaced; S8 owns the citation UX and its contract). mypy reports only the repo’s pre-existing uploaded_at overload note (CI does not enforce mypy), with no new error introduced.

One sentence to remember: a citation is no longer a page number but a typed segment — kind first, then coordinate — so a slide cites as a slide and a cell cites as a cell, while every page that ever existed still cites as a page through the resolver’s fallback.