RFD-DOC-002 S2 · pramaan-functions · office docs → render queue via a new
document.render.requested event type.
WHY this exists — the catastrophe it prevents.
S2 already shipped the render worker: a hardened LibreOffice Lambda that turns a
.docx/.pptx into a PDF the extraction stage can read. It is LIVE on staging.
But nothing fed it. Every clean upload — Word doc or PDF alike — was enqueued as
document.ingest.requested and went straight to extraction. A Word doc has no text layer the
extractor can read, so it hit the router with no render anchor and died at unreadable.
The entire render path was dead code: a fully-built, deployed, paid-for Lambda that no message could ever
reach. A lawyer uploading a brief as .docx — the single most common legal file — got nothing.
The malware-scan gate already decides "this upload is clean → hand it to the pipeline." We taught that hand-off to look at the file type: Office docs take a detour through the render stage first; everything else goes straight to extraction as before. One new event type carries the detour.
clean upload (malware gate, apply_scan_result)
│
┌─────────────────┴──────────────────┐
office? YES office? NO
│ │
document.render.requested document.ingest.requested
│ (→ render queue) │ (→ ingest queue)
▼ ▼
LibreOffice render worker extraction worker
│
renders PDF, then _emit_onward:
document.ingest.requested ───────────────────▶ extraction worker
(DIFFERENT idempotency key → new row → no loop)
| Part | What changed | Why titled by its reason |
|---|---|---|
| The event type | document.render.requested added to the DocumentIngestEventType Literal,
the DOCUMENT_INGEST_EVENT_TYPES tuple, and a new RENDER_RELAY_EVENT_TYPE const. |
The outbox relay already routes by event_type. A third type is the smallest possible
seam that keeps the render hop separate from the ingest hop without a second table. |
| The relay leg | relay_pending_document_render_events drains render rows to
resolved_document_render_queue_url, with its own
pramaan.document_render_relay.duration metric + DOCUMENT_RENDER_RELAY_BATCH span.
It runs inside the same outbox-relay Lambda as the ingest leg. |
One Lambda, granted SendMessage on both queues, drains both legs per invocation — no new Lambda to provision, no new schedule. Per-leg metric so render latency alarms independently. |
| The routing decision | In apply_scan_result (the malware gate), route by the re-hashed, magic-byte-detected
MIME: office & no render anchor → render.requested, else ingest.requested. |
The detected MIME is authoritative over the client-declared content type, and it is only known here — this is the one place the file's true type and the enqueue both live. |
| The CHECK constraint | Migration 0140 extends document_ingest_outbox_event_type_valid to admit the
third value. Single head, chained off a3f7c2e9b105. |
The constraint is the DB's guarantee that no typo'd event type ever lands; it must learn the new vocabulary before any row carries it. |
The fear with "render, then send it onward to extraction" is a cycle: render → extraction enqueue → malware gate sees an office doc again → render → … forever. Two independent guards make that impossible.
Guard 1 — different idempotency key. The outbox idempotency key is
(event_type, firm_id, document_id). The render worker's _emit_onward emits
document.ingest.requested (the default) — a different event type than the
document.render.requested row that drove the render. So they resolve to two distinct outbox
rows; the onward hop never collides with, or re-triggers, the inbound render request. _emit_onward
was left exactly as S2 shipped it — that two-hop is intended, not a bug to "fix".
Guard 2 — the render-anchor belt. The malware gate emits render.requested only when
rendered_pdf_key IS NULL. A freshly-stored doc always satisfies this; a doc that already carries
a render anchor is routed to extraction instead. Even if the gate somehow re-ran on a rendered doc, it could
never re-route it into the render stage.
render.requested key = document_ingest:sha256({"document.render.requested", firm, doc})
ingest.requested key = document_ingest:sha256({"document.ingest.requested", firm, doc})
└── different event_type → different key → no collision
The renderable-format set must mean the same thing on both edges: the malware producer (which
decides what to send to render) and the render consumer (which decides what LibreOffice opens). If
those two sets ever drifted, a format the gate sent to render could be one the engine never renders —
stranding the document. So RENDERABLE_OFFICE_CONTENT_TYPES + is_office_renderable
now live in one shared module, src/shared/document/render_formats.py, imported by both. The
render engine re-exports them so its existing callers are untouched.
| Test | Proves |
|---|---|
test_office_doc_routes_to_render_requested | A clean .docx at stored → document.render.requested. |
test_non_office_doc_routes_to_ingest_requested | A clean PDF → document.ingest.requested (extraction, unchanged). |
test_existing_render_anchor_never_re_routed_to_render | Loop guard: an office doc with a render anchor → ingest.requested, never render. |
test_render_relay_drains_render_rows_to_render_queue | The render relay sends render.requested rows to the render queue. |
test_render_and_ingest_relays_route_by_event_type_no_crosstalk | Each leg drains ONLY its own rows — no cross-talk in the shared outbox. |
test_clean_office_render_anchors_and_forwards (render worker integration) | The render worker emits onward as ingest.requested — the two-hop completes, no loop. |
test_render_and_ingest_idempotency_keys_differ_for_same_document | The substrate guarantee: the two hops resolve to distinct outbox rows. |
document.render.requested — lets the
malware gate detour office docs through the render worker before extraction, and because that worker emits
onward under a different idempotency key, the two-hop can never loop.