From 7e30f85c0b299024ee20210ecc326154dd168e39 Mon Sep 17 00:00:00 2001 From: zak Date: Thu, 23 Jul 2026 12:17:55 +0100 Subject: [PATCH 1/3] docs(ai-transport): add OpenAI Responses codec pages Add a getting-started walkthrough and a framework page for the OpenAI Responses codec (ResponsesCodec) from @ably/ai-transport/openai. - Getting started: build a Next.js chat app that streams the Responses event stream over a durable session using the generic createAgentSession and core React hooks, with cancellation and multi-device sync. - Framework page: what the Responses API brings, what AI Transport adds, the run.pipe connection point, and the server-side tool loop. - Add both pages to the AI Transport nav (OpenAI first under By SDK and first in Frameworks). - Remove the getting-started/openai redirect from the Vercel AI SDK page now that it resolves to a real page. --- src/data/nav/aitransport.ts | 8 + .../docs/ai-transport/frameworks/openai.mdx | 114 ++++++++ .../ai-transport/getting-started/openai.mdx | 274 ++++++++++++++++++ .../getting-started/vercel-ai-sdk.mdx | 1 - 4 files changed, 396 insertions(+), 1 deletion(-) create mode 100644 src/pages/docs/ai-transport/frameworks/openai.mdx create mode 100644 src/pages/docs/ai-transport/getting-started/openai.mdx diff --git a/src/data/nav/aitransport.ts b/src/data/nav/aitransport.ts index 823fd14135..e3db441a65 100644 --- a/src/data/nav/aitransport.ts +++ b/src/data/nav/aitransport.ts @@ -23,6 +23,10 @@ export default { name: 'Core SDK', link: '/docs/ai-transport/getting-started/core-sdk', }, + { + name: 'OpenAI', + link: '/docs/ai-transport/getting-started/openai', + }, { name: 'Vercel AI SDK', link: '/docs/ai-transport/getting-started/vercel-ai-sdk', @@ -110,6 +114,10 @@ export default { { name: 'Frameworks', pages: [ + { + name: 'OpenAI', + link: '/docs/ai-transport/frameworks/openai', + }, { name: 'Vercel AI SDK UI', link: '/docs/ai-transport/frameworks/vercel-ai-sdk-ui', diff --git a/src/pages/docs/ai-transport/frameworks/openai.mdx b/src/pages/docs/ai-transport/frameworks/openai.mdx new file mode 100644 index 0000000000..d24274bd01 --- /dev/null +++ b/src/pages/docs/ai-transport/frameworks/openai.mdx @@ -0,0 +1,114 @@ +--- +title: "OpenAI Responses" +meta_description: "How Ably AI Transport integrates with the OpenAI Responses API. The ResponsesCodec encodes the Responses event stream onto an Ably channel, and toResponsesInput feeds the conversation back to the model." +meta_keywords: "AI Transport, OpenAI, Responses API, ResponsesCodec, toResponsesInput, function calling, reasoning, durable sessions, server" +intro: "The OpenAI Responses API streams model output as typed events over HTTP. AI Transport's ResponsesCodec encodes that stream onto an Ably channel, so the same server code feeds durable sessions instead of an ephemeral HTTP response." +--- + +The [OpenAI Responses API](https://platform.openai.com/docs/api-reference/responses) is OpenAI's interface for calling a model and getting back a stream of typed events: output text, reasoning, refusals, function-call arguments, and lifecycle events. AI Transport writes that stream to an Ably channel, so reconnects, multi-device, and bidirectional control come for free. + +The `ResponsesCodec` from `@ably/ai-transport/openai` passes each raw Responses event through to the channel and reassembles the conversation on the client. There is no OpenAI-specific transport, so you use the generic [`createAgentSession`](/docs/ai-transport/api/javascript/core/agent-session) from `@ably/ai-transport` and pass it the codec. + +To build a working app with this codec, see [Get started with OpenAI](/docs/ai-transport/getting-started/openai). + +## What the OpenAI Responses API brings + +The `openai` npm package owns the model call and the typed event model: + +| Capability | Description | +| --- | --- | +| Responses streaming | `client.responses.create({ stream: true })` returns a stream of `ResponseStreamEvent`s: output-text deltas, reasoning, refusals, function-call arguments, and item lifecycle events. | +| Server-side tools | You advertise function tools on the request. The model emits function calls, you run them, and you feed the outputs back on the next Responses API call. | +| Reasoning models | Reasoning items carry the model's summarised thinking, and `encrypted_content` for the no-store, zero-data-retention round trip. | +| Round-trippable items | Each output item the codec stores is also valid Responses API input, so the conversation feeds the next turn without conversion. | +| Typed SDK | The official SDK owns auth, the HTTP call, streaming, and the Responses type definitions. | + +## What AI Transport adds + +AI Transport adds to the Responses API, writing the model stream to a durable session that outlives any single connection: + +| Capability | Description | +| --- | --- | +| Durable sessions | Tokens flow through an Ably channel that outlives any single connection. A client reconnects and resumes from where it left off. | +| Multi-device sync | Every device subscribed to the session sees the same conversation in realtime. | +| Bidirectional control | Cancel, steer, and interrupt the agent from any client. No separate control channel. | +| Active run tracking | `view.runs()` exposes which clients have Runs streaming and which Runs are in progress. | +| Conversation branching | Edit and regenerate create forks in the conversation tree, not destructive replacements. | +| History and replay | Load the full conversation on reconnect, page refresh, or new device join. | +| Token compaction | Reconnecting clients receive accumulated responses, not a replay of every token. | + +These are the same capability bullets used on the [Vercel AI SDK Core](/docs/ai-transport/frameworks/vercel-ai-sdk-core) page; the session surface is the same whichever model layer you integrate against. + +## Where they connect + +On the server, AI Transport swaps the HTTP response sink for `Run.pipe()`. Without it, the OpenAI SDK gives you no helper for forwarding a Responses stream over HTTP, so you serialise each event into a Server-Sent Events response by hand: + + +```javascript +const encoder = new TextEncoder(); +return new Response( + new ReadableStream({ + async start(controller) { + for await (const event of stream) { + controller.enqueue(encoder.encode(`data: ${JSON.stringify(event)}\n\n`)); + } + controller.close(); + }, + }), + { headers: { 'Content-Type': 'text/event-stream' } }, +); +``` + + +With AI Transport, a raw `ResponseStreamEvent` is already valid codec output and `run.pipe` takes the SDK's async iterable directly, so the same stream reaches a durable session in two lines, and every subscribed client resumes, syncs, and cancels it: + + +```javascript +const { reason } = await run.pipe(stream); +await run.end({ reason }); +``` + + +The integration has three pieces: + +1. The `ResponsesCodec` from `@ably/ai-transport/openai` encodes each `ResponseStreamEvent` as Ably messages and reassembles them into `OpenAIMessage`s on the client. It streams assistant text, refusals, reasoning, and function-call arguments, and repairs a client that joins a stream mid-way. The codec passes the raw events through, so the wire tracks OpenAI's own event model. +2. `createAgentSession({ client, channelName, codec: ResponsesCodec })` from `@ably/ai-transport` constructs the agent session bound to the channel from the invocation. +3. `run.pipe()` reads the model's event stream, encodes each event, and publishes the resulting Ably messages. `run.abortSignal` wires cancellation through from the client to the Responses API request. + +To feed the next turn, `toResponsesInput` flattens the conversation read from `run.view` back into the Responses `input` array. Each stored `OpenAIMessage` already holds valid Responses input items, so the flatten needs no conversion: + + +```javascript +import { toResponsesInput } from '@ably/ai-transport/openai'; + +while (run.view.hasOlder()) { + await run.view.loadOlder(); +} +const input = toResponsesInput(run.view.getMessages().map(({ message }) => message)); +``` + + +## Run server-side tools + +A Responses stream never carries a function call's *output*. OpenAI surfaces tool output only as model input on the next turn, so the codec adds one event of its own, `function_call_output`, for the agent to publish after it runs a tool. + +Server-executed tools do not suspend the Run. The agent runs an agentic loop: call the Responses API, and if the model emits function calls, run them, append the model's output items and the tool outputs to the input, and call it again. The loop continues until the model produces a reply with no tool calls. Each unit of work publishes under its own `run.pipe`, so a Run that calls one tool produces three messages: the turn that emitted the calls, the tool outputs, and the final text turn. + +For reasoning models, the loop must re-append the whole turn's output items, including the reasoning items that preceded a function call, since reasoning models expect that reasoning to travel with the call on the next request. The [runnable demo](https://github.com/ably/ably-ai-transport-js) implements the full loop. + + + +## Scope and trade-offs + +The `ResponsesCodec` is scoped to the streamed shapes the Responses API produces: assistant text, refusals, reasoning (summary and raw), and server-executed function calls. Client-side tools, hosted tools, and approval-gated tools are not yet supported; the codec exposes the user-message and regenerate input variants, with the tool variants to follow. + +The codec transmits the raw Responses events rather than a normalised abstraction. The wire therefore tracks OpenAI's own event model, which keeps stored items round-trippable to the Responses API but ties a conversation to the Responses shape. To integrate a different model provider behind one abstraction, use [Vercel AI SDK Core](/docs/ai-transport/frameworks/vercel-ai-sdk-core) instead. + +## Read next + +A Next.js chat app where: + +- Output from OpenAI's Responses API streams into a durable [session](/docs/ai-transport/concepts/sessions) that outlives the HTTP response. +- Closing a tab and reopening it resumes the in-progress response. +- A second tab on the same session sees the same conversation in realtime. +- A stop button cancels the in-progress [Run](/docs/ai-transport/concepts/runs). + +The [`ResponsesCodec`](/docs/ai-transport/frameworks/openai) maps the raw Responses event stream onto an Ably channel. You read the conversation with the core [`useView`](/docs/ai-transport/api/react/core/use-view) hook, which gives you branching, edit, regenerate, and pagination directly. + +## Prerequisites + +- Node.js 22 or later. +- An [Ably account](https://ably.com/sign-up) with an API key. +- An OpenAI API key set as `OPENAI_API_KEY`. + +## Install dependencies + +Install the AI Transport SDK, the Ably client, the OpenAI SDK, and Next.js: + + +```shell +npm install @ably/ai-transport ably openai next react react-dom +``` + + +## Set up authentication + +Create an auth endpoint at `/api/auth/token` that returns an Ably JWT to the client. The endpoint validates the user and signs a token with their client ID and the channel capabilities they need. See [Set up authentication](/docs/ai-transport/getting-started/authentication) for the full setup. + +The client below uses `authUrl: '/api/auth/token'` to fetch tokens from this endpoint. + +## Configure the channel rule + +AI Transport streams each response by appending tokens to a single channel message. That requires the **Message annotations, updates, deletes, and appends** channel rule (`mutableMessages`) on the namespace your conversations live on. + +In your Ably dashboard, enable **Message annotations, updates, deletes, and appends** on the `conversations` namespace. See [Configure the channel rule](/docs/ai-transport/getting-started/channel-rules) for the dashboard, Control API, and CLI steps. + + + +## Create the agent route + +Create `app/api/chat/route.ts`. The agent receives an [Invocation](/docs/ai-transport/concepts/invocations), creates an [AgentSession](/docs/ai-transport/api/javascript/core/agent-session) bound to the `ResponsesCodec`, starts a [Run](/docs/ai-transport/concepts/runs), rebuilds the conversation, opens a Responses stream, pipes it, and ends the Run: + + +```javascript +import { after } from 'next/server'; +import OpenAI from 'openai'; +import * as Ably from 'ably'; +import { createAgentSession, Invocation } from '@ably/ai-transport'; +import { ResponsesCodec, toResponsesInput } from '@ably/ai-transport/openai'; + +const ably = new Ably.Realtime({ key: process.env.ABLY_API_KEY }); +const openai = new OpenAI(); + +export async function POST(req) { + const invocation = Invocation.fromJSON(await req.json()); + + const session = createAgentSession({ + client: ably, + channelName: invocation.sessionName, + codec: ResponsesCodec, + }); + + await session.connect(); + + const run = session.createRun(invocation, { signal: req.signal }); + + after(async () => { + try { + // Load the full conversation from history before starting the Run. + // run.start() needs to have seen this Run's triggering input; draining it + // from history means run.start() resolves at once instead of waiting for + // that input to arrive live on the channel. + while (run.view.hasOlder()) { + await run.view.loadOlder(); + } + + await run.start(); + + // Flatten the conversation into the Responses `input` array. Each stored + // message already holds valid model input, so no conversion is needed. + const input = toResponsesInput(run.view.getMessages().map(({ message }) => message)); + + const stream = await openai.responses.create( + { model: 'gpt-5.5', input, stream: true }, + { signal: run.abortSignal }, + ); + + // The Responses stream is an async iterable, and each raw event is valid + // codec output, so pipe it straight through. run.pipe watches + // run.abortSignal, so a cancel ends the Run with no extra handling here. + const { reason } = await run.pipe(stream); + + await run.end({ reason }); + } catch (err) { + await run.end({ reason: 'error' }); + throw err; + } finally { + session.end(); + } + }); + + return Response.json({ runId: run.runId, invocationId: run.invocationId }); +} +``` + + + + +This route handles a single model turn. To configure server-side tools and run the agentic loop (model turn, run tools, continue), see [OpenAI Responses](/docs/ai-transport/frameworks/openai). + +## Create the chat component + +Create `app/chat.tsx`. The component reads the conversation with [`useView`](/docs/ai-transport/api/react/core/use-view) and sends a user turn with `ResponsesCodec.createUserMessage`. Because the core SDK never sends HTTP itself, the component POSTs to the agent endpoint after `view.send` resolves. + +An OpenAI message holds a list of `items`, so rendering flattens each message's text content parts: + + +```javascript +'use client'; + +import { useState } from 'react'; +import { useClientSession, useView } from '@ably/ai-transport/react'; +import { ResponsesCodec } from '@ably/ai-transport/openai'; + +async function wakeAgent(run) { + await fetch('/api/chat', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify(run.toInvocation().toJSON()), + }); +} + +// Flatten an OpenAI message's items into rendered text. +function messageText(message) { + let text = ''; + for (const item of message.items) { + if (item.type !== 'message') continue; + for (const part of item.content) { + if (part.type === 'output_text' || part.type === 'input_text') text += part.text; + } + } + return text; +} + +export function Chat() { + const [input, setInput] = useState(''); + + const { session } = useClientSession(); + const view = useView({ limit: 30 }); + const { messages, runOf } = view; + + // The Stop button shows only while the latest message's Run is streaming. + const latestRun = runOf(messages.at(-1)?.codecMessageId ?? ''); + const isStreaming = latestRun?.status === 'active'; + + const handleSubmit = async (e) => { + e.preventDefault(); + if (!input.trim()) return; + const text = input; + setInput(''); + const run = await view.send( + ResponsesCodec.createUserMessage({ + role: 'user', + items: [{ type: 'message', role: 'user', content: [{ type: 'input_text', text }] }], + }), + ); + await wakeAgent(run); + }; + + const stop = () => { + if (latestRun) void session.cancel(latestRun.runId); + }; + + return ( +
+ {messages.map(({ codecMessageId, message }) => ( +
+ {message.role}: {messageText(message)} +
+ ))} +
+ setInput(e.target.value)} placeholder="Type a message..." /> + {isStreaming ? ( + + ) : ( + + )} +
+
+ ); +} +``` + + +## Wire it together + +Create `app/page.tsx`. `Providers` sets up an authenticated Ably client. [`ClientSessionProvider`](/docs/ai-transport/api/react/core/providers) binds the channel and the `ResponsesCodec` into AI Transport. `ResponsesCodec` is a ready-made codec, so you pass it directly with no factory call. + +Update `channelName` to match a namespace with the AIT [channel rule](/docs/ai-transport/getting-started/channel-rules) configured: + + +```javascript +'use client'; + +import { useEffect, useState } from 'react'; +import * as Ably from 'ably'; +import { AblyProvider } from 'ably/react'; +import { ClientSessionProvider } from '@ably/ai-transport/react'; +import { ResponsesCodec } from '@ably/ai-transport/openai'; +import { Chat } from './chat'; + +function Providers({ children }) { + const [client, setClient] = useState(null); + + useEffect(() => { + const ably = new Ably.Realtime({ authUrl: '/api/auth/token', clientId: 'user' }); + setClient(ably); + return () => ably.close(); + }, []); + + if (!client) return null; + return {children}; +} + +export default function Page() { + const channelName = 'conversations:my-chat-session'; + return ( + + + + + + ); +} +``` + + +Run `npm run dev` and open `http://localhost:3000`. Open a second tab to the same URL; both tabs share the same durable session. + +## What is happening + +1. The user types a message. `view.send(...)` publishes the message on the channel and returns a `ClientRun`. The SDK does not POST to your agent endpoint itself. +2. Your client code calls `clientRun.toInvocation().toJSON()` and POSTs the resulting [`InvocationData`](/docs/ai-transport/concepts/invocations) to your agent endpoint. The body identifies the session and the input event the agent should respond to. +3. The agent endpoint creates an `AgentSession` bound to the `ResponsesCodec` and starts a Run. `run.start()` waits until it has seen the triggering input on the channel, whether loaded from history or arriving live. +4. The agent reads the full conversation from `run.view` and calls `toResponsesInput` to build the Responses `input` array. Each stored message is already valid Responses input. +5. The agent opens a streaming Responses API call and pipes the event stream through `run.pipe()`, which encodes each Responses event onto the channel. +6. Every client subscribed to the channel decodes the streamed events in realtime. `useView` re-renders as the visible Run accumulates OpenAI items. +7. If a client disconnects mid-stream, Ably resumes the subscription from the last serial on reconnect; the SDK rehydrates the view without losing tokens. + +## Understand the architecture + +The OpenAI SDK handles the model call and the typed event stream. AI Transport handles the durable session between agent and devices, encoding the Responses stream onto an Ably channel through the `ResponsesCodec`. See [OpenAI Responses](/docs/ai-transport/frameworks/openai) for how the codec works and how to run server-side tools. + +## Explore next + +- [OpenAI Responses](/docs/ai-transport/frameworks/openai): the codec, `toResponsesInput`, and the server-side tool loop. +- [Cancellation](/docs/ai-transport/features/cancellation): the stop button pattern and the agent-side `onCancel` authorisation hook. +- [Multi-device sessions](/docs/ai-transport/features/multi-device): open another tab to see realtime sync. +- [Branching, edit, and regenerate](/docs/ai-transport/features/branching): fork the conversation and navigate alternative branches. +- [Codecs](/docs/ai-transport/concepts/codecs): how a codec translates a framework's events into Ably messages. diff --git a/src/pages/docs/ai-transport/getting-started/vercel-ai-sdk.mdx b/src/pages/docs/ai-transport/getting-started/vercel-ai-sdk.mdx index 66db55fd7e..02de638275 100644 --- a/src/pages/docs/ai-transport/getting-started/vercel-ai-sdk.mdx +++ b/src/pages/docs/ai-transport/getting-started/vercel-ai-sdk.mdx @@ -7,7 +7,6 @@ redirect_from: - /docs/ai-transport/getting-started - /docs/ai-transport/getting-started/javascript - /docs/ai-transport/getting-started/anthropic - - /docs/ai-transport/getting-started/openai - /docs/ai-transport/getting-started/langgraph --- From 41f11e84170145d8ac3b8d489138569fd98e0fed Mon Sep 17 00:00:00 2001 From: zak Date: Thu, 23 Jul 2026 12:23:54 +0100 Subject: [PATCH 2/3] docs(ai-transport): note OpenAI codec is bundled too The codecs concept page implied the Vercel codec was the only bundled codec. Now that the OpenAI ResponsesCodec ships in the SDK, update the non-codec-specific pages so neither frames Vercel as the sole option. - codecs concept: the SDK bundles a Vercel codec and a ResponsesCodec, each covering its framework end-to-end; add an OpenAI Read next link; correct the intro from "an OpenAI completion" to "an OpenAI Responses event". - why overview: list OpenAI Responses events alongside the Vercel UIMessage as a codec-layer example. --- src/pages/docs/ai-transport/concepts/codecs.mdx | 5 +++-- src/pages/docs/ai-transport/why/index.mdx | 2 +- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/src/pages/docs/ai-transport/concepts/codecs.mdx b/src/pages/docs/ai-transport/concepts/codecs.mdx index 84c8598add..e116ddf25f 100644 --- a/src/pages/docs/ai-transport/concepts/codecs.mdx +++ b/src/pages/docs/ai-transport/concepts/codecs.mdx @@ -2,7 +2,7 @@ title: "Codecs" meta_description: "Understand codecs in AI Transport: the translation layer between your AI framework's events and Ably's messages." meta_keywords: "AI Transport, codec, TInput, TOutput, UserMessage, Regenerate, ToolResult, custom codec, Vercel" -intro: "A codec translates between your AI framework's events and Ably's messages. The same connection carries a Vercel UIMessage, an OpenAI completion, or a custom domain event without changing the SDK." +intro: "A codec translates between your AI framework's events and Ably's messages. The same connection carries a Vercel UIMessage, an OpenAI Responses event, or a custom domain event without changing the SDK." --- A codec is the translation layer between domain-specific message models and Ably's native message primitives. It defines how events in the application's domain (text deltas, tool calls, finish signals, or whatever the domain model requires) are encoded into Ably messages, and how incoming Ably messages are decoded back into domain events. @@ -76,11 +76,12 @@ const myCodec = { ``` -Most users don't write a custom codec; the bundled [Vercel codec](/docs/ai-transport/api/javascript/vercel/codec) covers the Vercel AI SDK end-to-end. See the [Codec API reference](/docs/ai-transport/api/javascript/core/codec) for the full interface. +Most users don't write a custom codec. The SDK bundles a [Vercel codec](/docs/ai-transport/api/javascript/vercel/codec) for the Vercel AI SDK and a [ResponsesCodec](/docs/ai-transport/frameworks/openai) for the OpenAI Responses API, each covering its framework end-to-end. See the [Codec API reference](/docs/ai-transport/api/javascript/core/codec) for the full interface. ## Read next + +The examples above use the Vercel codec, but the OpenAI Responses codec supports the same tool surface: server-executed function calls, client-executed tools, tool failures, and human approvals. The suspend and resume mechanics live in the transport, so they are identical for both codecs. The wire types and the factory payload field names differ. + +The OpenAI codec expresses tool state against the Responses types, so a tool call is a `function_call` item and its result is a `function_call_output` item. The client factories take snake_case payloads keyed by `call_id`, and you address each to the assistant message that holds the call: + + +```javascript +import { ResponsesCodec } from '@ably/ai-transport/openai'; + +// A client-run tool succeeded. +await view.send(ResponsesCodec.createToolResult(codecMessageId, { call_id, output }), { runId }); + +// A client-run tool failed. The message becomes the output the model sees next turn. +await view.send(ResponsesCodec.createToolResultError(codecMessageId, { call_id, message }), { runId }); + +// A user approved or denied a gated tool. A denial resolves entirely on the client. +await view.send(ResponsesCodec.createToolApprovalResponse(codecMessageId, { call_id, approved, reason }), { runId }); +``` + + +The Responses `function_call_output` item has no field for an approval decision or an error, so the codec holds that render-only state on `OpenAIMessage.toolCallStates`, a map keyed by `call_id`. `toResponsesInput` never reads it, so it cannot reach the model. See [OpenAI Responses](/docs/ai-transport/frameworks/openai) for the agentic loop and the approval-request output. + ## History persistence Tool invocations and results are part of the channel's message history. When a client reconnects or a late joiner loads the conversation, tool activity is replayed along with text messages. The view reconstructs tool state so the UI shows the correct status: pending, complete, or failed. diff --git a/src/pages/docs/ai-transport/frameworks/openai.mdx b/src/pages/docs/ai-transport/frameworks/openai.mdx index d24274bd01..f6a81cc242 100644 --- a/src/pages/docs/ai-transport/frameworks/openai.mdx +++ b/src/pages/docs/ai-transport/frameworks/openai.mdx @@ -1,7 +1,7 @@ --- title: "OpenAI Responses" meta_description: "How Ably AI Transport integrates with the OpenAI Responses API. The ResponsesCodec encodes the Responses event stream onto an Ably channel, and toResponsesInput feeds the conversation back to the model." -meta_keywords: "AI Transport, OpenAI, Responses API, ResponsesCodec, toResponsesInput, function calling, reasoning, durable sessions, server" +meta_keywords: "AI Transport, OpenAI, Responses API, ResponsesCodec, toResponsesInput, function calling, client-side tools, tool approval, reasoning, durable sessions, server" intro: "The OpenAI Responses API streams model output as typed events over HTTP. AI Transport's ResponsesCodec encodes that stream onto an Ably channel, so the same server code feeds durable sessions instead of an ephemeral HTTP response." --- @@ -96,13 +96,34 @@ Server-executed tools do not suspend the Run. The agent runs an agentic loop: ca For reasoning models, the loop must re-append the whole turn's output items, including the reasoning items that preceded a function call, since reasoning models expect that reasoning to travel with the call on the next request. The [runnable demo](https://github.com/ably/ably-ai-transport-js) implements the full loop. - +## Run client-side tools and approvals + +The codec also carries the client-driven half of tool calling: a client executes a tool in the browser and publishes the result, reports a tool failure, or answers a human approval prompt. The suspend and resume mechanics belong to the transport, so they work the same way they do for the Vercel codec. The agent calls `run.suspend()` to wait for a client, the client publishes its input on the same `runId`, and a continuation resumes the Run. + +`ResponsesCodec` exposes the full well-known factory set. You address each client input to the assistant message that holds the `function_call`, and key each payload by the OpenAI snake_case `call_id`: + + +```javascript +import { ResponsesCodec } from '@ably/ai-transport/openai'; + +// A client-run tool succeeded. +await view.send(ResponsesCodec.createToolResult(codecMessageId, { call_id, output }), { runId }); + +// A client-run tool failed. The message becomes the output the model sees next turn. +await view.send(ResponsesCodec.createToolResultError(codecMessageId, { call_id, message }), { runId }); + +// A user approved or denied a gated tool. A denial resolves entirely on the client. +await view.send(ResponsesCodec.createToolApprovalResponse(codecMessageId, { call_id, approved, reason }), { runId }); +``` + + +To gate a tool on a human decision, the agent publishes the codec's own `tool-approval-request` output, carrying the `call_id`, the tool name, and the arguments so a client can render the prompt without the streamed `function_call`. + +The Responses `function_call_output` item has no field for an approval decision or an error, so the codec holds that render-only state out of band on `OpenAIMessage.toolCallStates`, a map keyed by `call_id`. `toResponsesInput` never reads that state, so it cannot reach the model. ## Scope and trade-offs -The `ResponsesCodec` is scoped to the streamed shapes the Responses API produces: assistant text, refusals, reasoning (summary and raw), and server-executed function calls. Client-side tools, hosted tools, and approval-gated tools are not yet supported; the codec exposes the user-message and regenerate input variants, with the tool variants to follow. +The `ResponsesCodec` covers the shapes the Responses API streams: assistant text, refusals, reasoning (summary and raw), and server-executed function calls. It also covers the client-driven tool surface: client-executed tools, tool failures, and human approvals. Hosted tools (web and file search, code interpreter, image generation, MCP, custom tools) are not yet supported. The codec transmits the raw Responses events rather than a normalised abstraction. The wire therefore tracks OpenAI's own event model, which keeps stored items round-trippable to the Responses API but ties a conversation to the Responses shape. To integrate a different model provider behind one abstraction, use [Vercel AI SDK Core](/docs/ai-transport/frameworks/vercel-ai-sdk-core) instead.