Skip to content

feat: correlate inference requests with session ids - #6863

Open
chenghao-mou wants to merge 1 commit into
mainfrom
chenghao/feat/LKINF-344-inference-session-ids
Open

feat: correlate inference requests with session ids#6863
chenghao-mou wants to merge 1 commit into
mainfrom
chenghao/feat/LKINF-344-inference-session-ids

Conversation

@chenghao-mou

@chenghao-mou chenghao-mou commented Aug 14, 2026

Copy link
Copy Markdown
Member

Inference gateway calls need a correlation key that follows each transport lifecycle. Session IDs are SDK-owned.

In-job correlation:

Component Session header (key: value) Session scope Per-request ID pattern
STT X-LiveKit-Session-ID: inference_<hex12> WebSocket Service-defined; fallback: session ID
TTS X-LiveKit-Session-ID: inference_<hex12> Pooled WebSocket inference_<hex12>_tts_<hex12> per synthesis
LLM X-LiveKit-Session-ID: inference_<hex12> HTTP attempt Service-defined; no derived SDK ID
EOT X-LiveKit-Session-ID: inference_<hex12> WebSocket inference_<hex12>_eot_<hex12> per prediction
Barge-in X-LiveKit-Session-ID: inference_<hex12> WebSocket created_at nanoseconds per frame; no string ID

Outside a job, no session header is sent and existing request-ID formats remain. Custom headers cannot replace the SDK-owned session ID.

Addresses LKINF-344

Initial prompt and agent context

Model: GPT-5.6

we need to pass the client side request id to inference as a session id header $grill-me

I am not asking about job, I am talking about inference stt/tts/llm/eot/bargein

we should change this a bit to make it semantic:

  • connection session id: generated once, use for the connection header (stt, tts, and llm http requests)
  • tts per request: reuse the session_id but append a suffix so as to correlate locally.

tts or eot

inference_ is enough

@chenghao-mou
chenghao-mou requested a review from a team as a code owner August 14, 2026 22:16

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 1 additional finding.

Open in Devin Review

@chenghao-mou
chenghao-mou requested a review from a team August 14, 2026 22:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants