Story 1857: the ZIP-expansion worker — a bundle becomes N first-class documents without ever bypassing the malware scanner or escaping its own folder

Why this exists (the catastrophe it prevents). Counsel almost never uploads one clean PDF; they drop a discovery-bundle.zip of 300 files. "Unpack it and treat each file as a document" sounds trivial, and that is exactly why it is dangerous. A naive recursive unzip is two well-known exploits waiting to happen. Zip-slip: an entry named ../../../../etc/passwd (or, in our world, ../../firms/OTHER_FIRM/...) writes outside the document prefix — a cross-firm write that shatters tenant isolation. Zip-bomb: a 42 KB archive that inflates to 4.5 PB exhausts the worker's memory and DLQs the queue. And the subtler one: if the worker writes a child object anywhere the malware-results parser does not recognise, or hands the child straight to extraction, that child's bytes were never scanned — the upload pipeline's whole scan-before-extract guarantee is silently bypassed for every file inside a ZIP.

Satellite view: where this sits in the pipeline

A ZIP container is a documents.documents row like any upload; it is scanned to stored. The readability router (a sibling story) classifies it as ARCHIVE_ZIP and enqueues a container pointer onto the zip-expand SQS queue instead of sending it to extraction. This Story is the consumer of that queue. Per record it: fetches the container's bytes from S3, streams them through the pure safety core, materialises each safe entry as its own child documents.documents row under raw/, writes a document_import_items ledger row for every entry (accepted zip_child or rejected + reason), and acks. It never enqueues extraction for a child — GuardDuty + the malware-results consumer do that, so the scan gate is never skipped.

container (stored)                          THIS WORKER                       per child, for free
  router: ARCHIVE_ZIP  --SQS pointer-->  fetch bytes from S3
                                          expand() one entry at a time
                                            +-- AcceptedChild --> new documents row (uploaded)
                                            |                     + bytes to firms/{f}/matters/{m}/
                                            |                       documents/{child}/raw/{safe}
                                            |                     + ledger zip_child (landed)        --> GuardDuty PUT scan
                                            |                                                            --> malware-results
                                            |                                                            --> stored -> extract
                                            +-- RejectedEntry --> ledger rejected + reason_code
                                          CorruptArchive --> container item rejected (no DLQ)

The decisions that matter

1. Sanitisation reuses the upload path verbatim — one definition of "safe name" (V-ZIPNAME)

Every entry name passes through DocumentService._safe_filename — the same helper presign uses — which strips directory components (split("/")[-1].split("\\")[-1]), allowlists [A-Za-z0-9._ -], and floors to "document". The key is then built by _build_object_key, the single source of truth for the raw/ grammar. So ../../etc/passwd, a/b/../../../c.txt, and /abs/evil.sh all collapse to a bare basename under the child's own UUID-keyed prefix — the final key always has exactly the prefix shape and a basename with no /, \, .., or NUL. The unsanitised original is preserved in the ledger's source_path for the Record display only, never in a key.

2. Caps are enforced while streaming, and every breach is a ledger fact — never a throw (the B-7 lesson)

The pure core reads each entry in 1 MiB chunks, aborting at the per-child ceiling so a lying header cannot smuggle an over-cap child past a cheap precheck. Each breach yields a RejectedEntry carrying a closed reason code, which becomes a rejected ledger item — the archive is still fully accounted, and the worker never raises on content. That is the same principle the extraction worker learned: a content failure that throws becomes a retry becomes a DLQ black hole. Here it is a clean, reason-coded row instead.

A nested ZIP is detected by magic bytes (PK\x03\x04), not the attacker-controlled extension. A wholly unreadable archive (cannot even be listed) is the one case that is not a per-entry reject — it raises CorruptArchiveError from the core so the worker records a container-level rejected ledger fact and acks, still never a DLQ.

3. Children re-enter the scan gate — the worker NEVER direct-enqueues extraction (§5.4)

Each child is created as a documents.documents row in processing_status = 'uploaded' and its bytes are written to its raw/ key, then the worker stops. GuardDuty scans every object under the managed prefix on PUT; the existing malware-results consumer fires per child and runs the normal CLEAN→stored (+ matter-scoped dedup) → extraction path. The child inherits the container's stored W3C trace carrier, so its scan→stored hop continues the same trace_id — the whole expansion subtree is one trace. Direct child enqueue is forbidden by construction: there is no document.ingest.requested write in this worker at all.

4. Idempotent re-expansion (§5.6)

SQS is at-least-once, so a container can be redelivered. Before materialising an entry the worker checks the ledger for an existing item keyed on (container_document_id, source_path); a child already landed/accounted is counted but never re-created or re-written. Child object keys are UUID-keyed per child, so even colliding sanitised names never collide on disk.

5. Observability: one span per boundary, low-cardinality counts, continuous trace (ADR-124)

Two registry-approved manual span names were added to the closed ADR-124 enum: functions.document_zip_expand.batch (one per SQS invocation, links every record's stored trace context) and functions.document_zip_expand.process_record (CONSUMER kind, parented to the container's stored traceparent for continuity). The per-record span carries only safe, low-cardinality attributes — firm_id, matter_id, document.mime_type, and the numeric unpack counts document.zip.child_count / document.zip.rejected_count / document.zip.depth (added to the document-KPI allowlist, structurally excluded from X-Ray annotation promotion). No raw entry names or document bytes ever reach telemetry — reason codes are a closed enum. Queue backlog/in-flight visibility rides the consumer's emit_sqs_consume_duration + emit_sqs_message_oldest_age (same as extraction) plus the queue's native SQS CloudWatch metrics and the DLQ alarm owned by infra Chore C1; the worker does not poll the queue (Lambda must not).

Verification

The security core is proven directly on bytes; the orchestration over a fake sink; the telemetry contract against the registry.

All 18 new unit tests pass. ruff check is clean on every changed file; mypy reports no error in src/workers/document_zip_expand/ (the pre-existing SQLModel typing noise in untouched files is unchanged and out of scope; mypy is local-only); git diff --check is clean. No route, request/response DTO, or trust-boundary HTTP payload changed (the SQS envelope is an internal message, not an API DTO), so the OpenAPI snapshots are unchanged.

One sentence to remember: a ZIP becomes N first-class child documents whose names can never escape their own firm/matter prefix and whose bytes always re-enter the GuardDuty scan gate before extraction, while every bomb, traversal, or corrupt entry is an accounted, reason-coded ledger row instead of a crash or a DLQ.