Story 2162: Station 4 gets a running engine — and a wired fuel line

WHY. The CE-3 context_embed worker's code shipped earlier (src/workers/context_embed/handler.py + processor.py): given one per-document enrich kick off the context-enrich queue, it chunks the document's pages and embeds them into context.chunks. But two links were missing, and Station 4 (Context Engine enrichment) was dead on staging. First, the worker had no Lambda — a consumer with nothing draining its queue. Second, the queue URL was never injected into the two producers that enrich-kick — the registration consumer and app-api's inbox_service — so on a fresh upload the kick failed with InvalidAddress (enrich_kick_failed in the logs) and context.chunks stayed at zero rows. The catastrophe this Story prevents is a pipeline that reads as built but produces nothing: merged, tested embed code that never runs because it was never deployed, fed by a kick that can never reach a queue nobody told it about.

🛰️ Satellite — the one sentence

A new deploy workflow (deploy-context-embed-worker.yml) creates the pramaan-context-embed Lambda in the staging VPC and event-source-maps it to the context-enrich queue with cross-region bedrock:InvokeModel for cohere.embed-v4 in us-east-1; and both enrich-kick producers (registration consumer + app-api) now inject CONTEXT_ENRICH_QUEUE_URL from a single infra-published SSM param — so the kick reaches a real queue and a real worker drains it.

✈️ Airplane — the same delivery truck, plus a queue and a passport

The document workers already ride a well-worn delivery route: package src/ + otel_wrapper.py into a zip, attach the shared deps layer and the ADOT OTel layer, drop it into the staging VPC so it can reach RDS, stamp it with the commit SHA. The embed worker rides that same truck — but, unlike the direct-invoke backfill sibling (Story-2157), it is an SQS consumer, so it needs the same two extras the registration consumer carries: an event-source mapping that pulls records off the enrich queue, and a role scoped to consume that queue. It adds one thing no document worker needs: a cross-region passport — cohere.embed-v4 is not offered in ap-south-1, so the embedder invokes Bedrock in us-east-1, and the role grants bedrock:InvokeModel on that region's model. It needs no S3 and no AgentCore: it reads pages from the DB and writes vectors to the DB, nothing else.

🔬 Microscopic — the moving parts

1. One source of truth for the queue URL — no per-workflow divergence

Three workflows need the enrich queue URL: the embed worker (to wire its ESM), and the two producers (to send the kick). Hardcoding a queue name in three places is exactly the divergence Bug-2146 warns about — one edit and they silently disagree. Instead all three read the same infra-published SSM param (Story-2161):

aws ssm get-parameter --name /pramaan/staging/context/enrich-queue-url \
  --query Parameter.Value --output text 2>/dev/null || echo ""

Because CONTEXT_ENRICH_QUEUE_URL is an Optional setting, an absent param resolves to empty → None, and every deploy still succeeds. That is what lets this PR merge before infra applies: the producers degrade to a best-effort (log-and-swallow) kick, and the worker deploy simply skips its ESM until the queue exists.

2. The producers — URL and permission, the real enrich_kick_failed fix

A kick needs two things the bug was missing: the queue URL (the InvalidAddress cause) and sqs:SendMessage on it. The registration consumer's role is fully managed by its own deploy workflow, so both go there — the URL into its env, and a new IAM statement:

{"Sid":"EnrichKick","Effect":"Allow","Action":["sqs:SendMessage"],
 "Resource":"arn:aws:sqs:${AWS_REGION}:${ACCOUNT_ID}:pramaan-context-enrich*"}

The name-pattern resource is stable and pre-provisionable — no queue ARN to resolve, so the policy is correct even before the queue exists. app-api also kicks (from inbox_service), but it runs on the shared, infra-owned pramaan-lambda-execution-role; this workflow can only inject its URL, so the app-api SendMessage grant on the enrich queue is left to infra (Story-2161) and is called out in the PR.

3. The worker role — consume the queue, invoke Bedrock cross-region

Least-privilege, four statements: Logs, SQS-consume on pramaan-context-enrich*, KMS (to decrypt the DB URL, region-pinned to ap-south-1), and the passport:

{"Sid":"BedrockEmbed","Effect":"Allow","Action":["bedrock:InvokeModel"],
 "Resource":["arn:aws:bedrock:us-east-1::foundation-model/cohere.embed-*"]}

The embed-* wildcard mirrors embedder.py's promise that a model swap is one config line — a v4→successor bump needs no IAM change. Network reachability to us-east-1 Bedrock from the ap-south-1 VPC is infra's concern (VPC endpoint / egress), not this workflow's.

4. The ESM — wired only when the queue resolves

The wiring step mirrors the registration consumer's: raise the queue's VisibilityTimeout to ≥ the 300 s function timeout (the EPIC-1489 ESM precondition), then create-or-update a batch-size 1, MaximumConcurrency=5, ReportBatchItemFailures mapping with an IAM-propagation retry loop. The one difference: if CONTEXT_ENRICH_QUEUE_URL is empty (queue not yet provisioned), the step logs and exits 0 instead of failing — so the first, pre-infra deploy is green and a redeploy after Story-2161 applies wires the mapping.

5. The smoke test — an empty batch proves the shell

Same shape as the registration consumer: an empty {"Records":[]} invoke must return {"batchItemFailures":[]}, proving the artifact imports, the cached event loop spins, and the handler is reachable on the live shell — without touching any firm's data. The real end-to-end evidence (a fresh upload → enrich_kicked → context.chunks rows > 0) is run separately, after Story-2161 provisions the queue, and recorded on the issue.

What this Story deliberately does NOT do

Left outWhy
Provision the enrich queue / DLQ / infra IAMTerraform-owned in pramaan-infra (Story-2161); this repo owns only the function artifacts + their own roles.
app-api SendMessage IAM on the enrich queueapp-api runs on the shared infra-owned execution role; the grant is infra's (Story-2161). This workflow injects only the URL.
us-east-1 Bedrock VPC endpoint / egressNetworking substrate is Terraform-owned; the workflow grants only the IAM to invoke.
S3 / AgentCore permissionsThe embed worker reads pages from RDS and writes vectors to RDS; it touches neither.
Recall in one sentence: Story-2162 stands up the pramaan-context-embed Lambda as an enrich-queue consumer with a cross-region Cohere-embed grant, and injects the one infra-published enrich queue URL into both kick producers (registration consumer + app-api) — so the Station 4 kick finally reaches a real queue and a real worker fills context.chunks.