Propagate W3C trace context through Python UDF server and AI clients MCDB-103303 #closes

[nkitsaini]

Summary

This change carries an incoming W3C trace context (traceparent) through Python UDF execution and forwards it to supported outbound AI requests. This makes it possible to follow one request across the upstream proxy, the UDF, and the downstream model endpoint using the same trace ID.

What changed

• The UDF server now extracts a valid incoming traceparent and keeps it active while async or sync UDF code runs.
• UDF request logs now include the trace ID for request start, timeout, cancellation, errors, and metrics.
• OpenAI and Azure embedding requests, including requests made with caller-provided clients, now receive the active trace context.
• Bedrock embedding requests now receive the active trace context.
• Async httpx clients now use an async request hook, fixing the TypeError previously raised by async chat calls such as ainvoke and abatch.
• Shared OpenTelemetry header helpers now live in singlestoredb/utils/otel.py.

If traceparent is missing or invalid, behavior is unchanged. The existing per-request request_id is also unchanged.

This uses only opentelemetry-api; it does not create spans or require an OpenTelemetry SDK or exporter. When opentelemetry-api is not installed, tracing remains a no-op. The package is included only in the test dependency group.

Compatibility note

When OpenAI or Azure embedding callers do not provide an HTTP client, the factory now creates sync and async httpx clients with a 600-second timeout and a 5-second connection timeout, matching the chat factory. These clients use httpx default connection-pool limits rather than the OpenAI SDK defaults.

Test Plan

• Added coverage for trace propagation and trace-aware logging in async and sync UDFs, including missing and invalid headers.
• Added coverage for default and caller-provided OpenAI clients, Azure-compatible clients, Bedrock requests, and async chat requests.
• Verified that async chat calls no longer fail because of a synchronous event hook.
• Ran the targeted test suite: 40 tests passed.
• Ran pre-commit checks successfully.
• Manually verified propagation with the real libraries for sync and async embeddings, chat ainvoke, and chat batch: all five requests sent the incoming traceparent unchanged.

JIRA Issues

Closes MCDB-103303

[Copilot]

🟢 Approval recommended

The propagation paths, optional-dependency behavior, logging integration, and synchronous/asynchronous clients are implemented consistently and covered by targeted tests.

0 open findings

What changed in this PR

Adds end-to-end W3C trace-context propagation through UDF execution and outbound AI requests.

Changes:

• Extracts incoming trace context and adds trace IDs to UDF logs.
• Adds reusable synchronous/asynchronous HTTP header-injection hooks.
• Extends AI-client tests and test dependencies for OpenTelemetry.

File	Description
singlestoredb/​utils/​otel.py	Provides reusable OpenTelemetry injection hooks.
singlestoredb/​functions/​ext/​asgi.py	Propagates request trace context through UDF execution.
singlestoredb/​functions/​ext/​utils.py	Permits trace IDs in JSON logs.
singlestoredb/​ai/​chat.py	Uses shared sync/async tracing hooks.
singlestoredb/​ai/​embeddings.py	Adds tracing to OpenAI, Azure, and Bedrock requests.
singlestoredb/​tests/​test_udf_trace_context.py	Tests UDF propagation and logging.
singlestoredb/​tests/​test_embeddings.py	Tests embeddings-client propagation.
singlestoredb/​tests/​test_ai_chat.py	Tests asynchronous chat-client propagation.
pyproject.toml	Adds OpenTelemetry API to test dependencies.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.