Skip to content

feat: conversational context (userId/conversationId/threadId) on toAi() routes - #681

Merged
lmajano merged 2 commits into
developmentfrom
claude/ai-routing-thread-context
Aug 17, 2026
Merged

feat: conversational context (userId/conversationId/threadId) on toAi() routes#681
lmajano merged 2 commits into
developmentfrom
claude/ai-routing-thread-context

Conversation

@lmajano

@lmajano lmajano commented Aug 17, 2026

Copy link
Copy Markdown
Member

Description

toAi() works great for single-shot inference but had no way to carry conversational identity across a multi-turn exchange - no userId, conversationId, or threadId. This adds all three to the invoke/stream/batch sub-routes toAi() registers.

Jira Issues

COLDBOX-1417

Type of change

  • Bug Fix
  • Improvement
  • New Feature
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Checklist

  • My code follows the style guidelines of this project cfformat
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

What changed

A new shared resolveAiContext( body ) private method on Router.cfc, called by all three of toAi()'s runnable-invoking sub-routes:

  • userId - the request body's userId if provided, else the framework's own Controller.getUserSessionIdentifier() (session id, or a cookie/URL-token-based tracking id as a fallback, or a per-request id as a last resort) - so every call is attributable to someone even when the caller doesn't manage its own user identity.
  • conversationId - passed through as-is when the caller supplies one. No default is generated - an absent conversationId means the caller isn't tracking conversations, and inventing one would imply a continuity that doesn't exist.
  • threadId - passed through if supplied, otherwise a new one is generated (createUUID()). Always present in the result, so a follow-up call can continue the same thread whether the caller supplied a threadId or a new one had to be minted.

All three resolved values are merged into the options struct passed to the runnable's run()/stream() calls (options.userId, options.conversationId, options.threadId), so an IAiRunnable implementation sees them without any interface changes.

Request/response shape

// POST /api/chat/invoke
// { "input": "hi", "threadId": "t-123" }

// → runnable.run( "hi", {}, { userId: "<session id>", threadId: "t-123" } )
// → { "output": ..., "success": true, "threadId": "t-123" }
// → response header: X-Thread-Id: t-123

threadId is echoed back three ways so it's usable from any client:

  • On the JSON response body (invoke/batch)
  • As an X-Thread-Id response header (all three sub-routes)
  • As a leading event: thread SSE frame on /stream, sent before the runnable's own chunks - browser EventSource clients can't read response headers, so this is the only way they learn a server-generated threadId in time to persist it

batch resolves context once per request and shares it across every item in inputs[], consistent with how params/options already work for batch.

Example

// A runnable reading the resolved context
component implements="bxModules.bxai.models.runnables.IAiRunnable" {
    function run( input, params = {}, options = {} ){
        var thread = conversationStore.loadOrCreate(
            userId         = options.userId,
            conversationId = options.conversationId ?: "",
            threadId       = options.threadId
        );
        return chatModel.reply( thread, input );
    }
}

Files touched

  • system/web/routing/Router.cfc - resolveAiContext() private method; wired into the invoke, stream, and batch sub-route response closures registered by toAi(); docblock updated with the new request body fields and response/header behavior
  • tests/specs/web/routing/RouterAITest.cfc - new specs covering resolveAiContext()'s userId default/passthrough, conversationId passthrough/absence, and threadId passthrough/generation (via makePublic(), matching this file's existing testing conventions)

Testing notes

RouterAITest.cfc extends BaseModelTest (skip="notBoxlang", matching the existing AI/MCP routing test file) and exercises resolveAiContext() directly via makePublic(), with no servlet/database dependency, so it's runnable in CI as-is. The sandbox this PR was authored in has no reachable test database, so the full TestBox HTTP runner couldn't be exercised locally regardless of engine; instead, Router.cfc was verified to still parse and load correctly after every edit (arrow-function closure bodies are parsed eagerly in CFML/BoxLang, so a syntax error in the edited invoke/stream/batch closures would have surfaced immediately on instantiation, even without executing them), and resolveAiContext() was confirmed present on the compiled class via getMetadata().

…() routes

toAi() previously only forwarded input/params/options to the runnable, with
no way to carry conversational identity across a multi-turn exchange. Adds
a shared resolveAiContext() used by the invoke/stream/batch sub-routes:

- userId: request body's userId if provided, else Controller's own
  request/session tracking identifier (getUserSessionIdentifier())
- conversationId: passed through only if the caller supplies one - no
  default is generated
- threadId: passed through if supplied, otherwise generated - always
  returned to the caller (JSON response on invoke/batch, X-Thread-Id header
  on all three, plus a leading SSE "thread" frame on stream, since
  EventSource clients can't read response headers) so a follow-up call can
  continue the same thread

All three resolved values are merged into the options struct passed to the
runnable's run()/stream() calls, so a handler implementation sees them at
options.userId/options.conversationId/options.threadId.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCmPkBvNZZhU6iuDcG4NR
@github-actions

github-actions Bot commented Aug 17, 2026

Copy link
Copy Markdown

Test Results

0 tests  ±0   0 ✅ ±0   0s ⏱️ ±0s
0 suites ±0   0 💤 ±0 
0 files   ±0   0 ❌ ±0 

Results for commit b414b4d. ± Comparison against base commit a85f7a5.

♻️ This comment has been updated with latest results.

Adobe's parser cannot handle a member method call chained directly onto a
parenthesized expression - ( body.options ?: {} ).append( aiContext, true )
crashed the compiler on adobe@2023/adobe@2025 CI with "Invalid CFML
construct". Same category of ACF parser limitation already hit once in this
file this session (array literals instead of a parenthesized Elvis
expression this time). Fixed by assigning to a local var first, then
calling .append() on the var, in all three sub-routes (invoke/stream/batch).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCmPkBvNZZhU6iuDcG4NR
@lmajano
lmajano merged commit 688739c into development Aug 17, 2026
23 checks passed
@lmajano
lmajano deleted the claude/ai-routing-thread-context branch August 17, 2026 02:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants