Expose OpenAI prompt cache request options - #6832
Open
brandon-julio-t wants to merge 1 commit into
Open
Conversation
🦋 Changeset detectedLatest commit: 7b0852e The changes in this PR will be included in the next version bump. This PR includes changesets to release 30 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
- New
CreateResponsefields — addedsafety_identifier(64-char max),prompt_cache_key,prompt_cache_retention("in_memory" | "24h"), andprompt_cache_options({ mode?: "implicit" | "explicit", ttl?: "30m" }) to the handwritten Responses schema. These flow automatically intoConfigvia the existingPartial<Omit<CreateResponse.Encoded, ...>>type and through to HTTP request bodies via the...apiConfigspread inmakeRequest. - Integration tests — one non-streamed test (GPT-5.5, legacy retention) and one streamed test (GPT-5.6, new cache options) verify all fields reach HTTP request bodies. The streaming test uses the new
makeStreamHttpTestLayerwhich exercises the realOpenAiClientand HTTP pipeline, a better integration path than the existing client-mockmakeStreamTestLayer. - Schema test — validates decode of all four fields and rejects
safety_identifierover 64 characters. - Test infrastructure —
MockOpenAiResponsegained an optionaleventsfield (backwards-compatible —bodywas always provided in existing calls) andmakeHttpClientroutes to SSE-format responses when events are present.
DeepSeek Pro (free via Pullfrog for OSS) (Kimi K2 not used — the program covers this model; add its provider key to run your pick) ｜ 𝕏
Contributor
Bundle Size AnalysisGenerated from PR build output; treat the content below as untrusted.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
RFC status
Implementation complete and tested. This PR is ready for review. It is framed as an RFC because feedback is requested on the public
OpenAiSchema.CreateResponse/OpenAiLanguageModel.Configshape, not because implementation work remains.Summary
Expose the following OpenAI Responses API request-level options through
OpenAiLanguageModel.Config:safety_identifierprompt_cache_keyprompt_cache_retention(in_memoryor24h)prompt_cache_options(modeandttl)The request builder already forwards schema-derived config, so the implementation is deliberately limited to the handwritten request schema, request/schema tests, and a patch changeset. Content-level
prompt_cache_breakpointis not included; that remains #6831.Design decisions
OpenAiSchema.CreateResponse, which is the source for the high-level model config and client request type.prompt_cache_options.modeas optionalimplicit | explicitandttlas optional30m, matching current GPT-5.6 Responses request semantics and defaults.prompt_cache_retention: in_memory | 24h.safety_identifier.@effect/ai-openai-compatunchanged: it uses a separate Chat Completions-compatible request surface and is not the Responses high-level config reported in this issue.Generated vs handwritten schema
The checked-in generated OpenAI request schema already contains
safety_identifier,prompt_cache_key, andprompt_cache_retention, but it does not contain the newerprompt_cache_options. Current official OpenAI Responses documentation and the official Node SDK do documentprompt_cache_optionsfor GPT-5.6+.This PR does not hand-edit
Generated.tsor add a speculative codegen patch. The generated client is not the source ofOpenAiLanguageModel.Config; keeping this change in the handwritten Responses schema provides the requested high-level support without creating a broad generated diff. Generated parity can follow when the configured Stainless schema includes the field, or via a maintainer-preferred codegen patch.Focused RFC questions
prompt_cache_optionsremain an inline nested schema, consistent with nearby request options, or be exported as a named public schema?nullto mirror raw OpenAI SDK types, or remain optional non-null values consistent with the existing handwritten config style?Generated.tsunchanged preferable until the configured upstream schema catches up, or would maintainers prefer a targeted codegen patch now?Verification
pnpm lint-fixpnpm test --run packages/ai/openai/test/OpenAiLanguageModel.test.ts packages/ai/openai/test/OpenAiSchema.test.ts— 61 tests passedpnpm checkPositive tests verify that all supported fields survive high-level model config and reach real HTTP request bodies:
stream: trueSchema coverage also verifies the supported option shapes and rejects safety identifiers longer than 64 characters.
Tradeoffs
prompt_cache_options; that limitation is explicit rather than hidden by guessing at generated output.Closes #6830