Story-1139 — Continue the trace from SQS through Bedrock
Implements pramaan-docs ADR-104 for pramaan-agents: the already-attached ADOT Lambda layer is the ONE tracing system; this Story makes the agents Lambda continue the trace that pramaan-functions started.
The trace journey
HTTP request (traceparent + x-pramaan-correlation-id)
→ pramaan-functions route span
→ SQS enqueue: message attributes traceparent / tracestate / correlation_id
→ agents Lambda: consume_record() extracts context
→ "pramaan-agents-runs-{env} process" CONSUMER span (parent = producer span)
→ Bedrock / SQS / API GW management spans from the ADOT layer's botocore
instrumentation, parented under the consumer span
Middleware home: the single SQS record-consume wrapper
-
src/pramaan_agents/runtime/sqs_consume.pyis the repo's middleware home (pramaan-docs ADR-065 / ADR-104). ALL trace-context extraction lives here — no per-handler or per-tool span code. - Single-record events: the extracted W3C context is the parent of the consumer span — the upstream trace continues.
- Batches (>1 record): each record's span is a root with a span link to its producer context, per the OTel messaging semantic conventions.
-
Old-producer messages (no attributes): new root trace, minted
correlation_id, zero errors — backward compatible by construction. -
Only
opentelemetry-apiis used against the global provider. The ADOT layer owns the TracerProvider; no SDK setup exists in this repo (exactly ONE TracerProvider at runtime).
Correlation contract
-
Resolution order: SQS message attribute
correlation_id>AgentCommand.correlation_idenvelope field (new, optional) > minted UUID v4. -
Set as the
correlation_idspan attribute on the consumer span (queryable in Transaction Search) and bound into a contextvar for the logger.
Structured logs (PRAMAAN log shape v1)
-
stdlib logging replaced by Powertools Logger +
PramaanLogFormatter(src/pramaan_agents/observability/logger.py) — same line shape as pramaan-functions. -
Every line carries
correlation_id, plustrace_id/span_idwhenever a span is active; sensitive keys and JWT/AWS-key/url-password strings are redacted.
Kill switch and SnapStart safety
-
OTEL_SDK_DISABLED=true(function config only) — code never depends on tracing being enabled; verified by a clean-interpreter test with the switch set. -
Bedrock (
BedrockModel) and DB clients stay lazy per pramaan-docs ADR-102 — no import-scope network state; layer auto-instrumentation covers boto3/Bedrock spans.
Dependency hygiene
aws-xray-sdkremoved (dropped Powertoolstracerextra) — ADR-104 guardrail.- Unused
structlogremoved;opentelemetry-apideclared explicitly.
Test evidence
| Behavior | Test |
|---|---|
| Trace continues from message attributes | tests/observability/test_sqs_consume.py, test_handler_trace.py |
| Backward compat (old producer) | test_missing_attributes_starts_new_root_trace, test_old_producer_record_without_attributes_still_processes |
| Batch span links | test_batch_records_use_span_links_not_parent, test_multi_record_batch_uses_links |
| Correlation precedence + envelope field | test_correlation_*, tests/runtime/test_payloads_correlation.py |
| Log shape with trace_id/span_id/correlation_id | tests/observability/test_logger.py |
| Kill switch | test_kill_switch_processing_works_without_tracing |