Add OpenAI prompt cache breakpoints - #6833
Open
brandon-julio-t wants to merge 1 commit into
Open
Conversation
🦋 Changeset detectedLatest commit: 41a6e19 The changes in this PR will be included in the next version bump. This PR includes changesets to release 30 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
PromptCacheBreakpointschema and type — added toOpenAiSchemawith the single valid value{ mode: "explicit" }; wired intoInputTextContent,InputImageContent, andInputFileContentasSchema.optionalKey.- Provider option augmentations —
SystemMessageOptions,UserMessageOptions,FilePartOptions, andTextPartOptionsall gainopenai.promptCacheBreakpointfollowing the established Anthropic/OpenRouter pattern. - Converter logic —
prepareMessagesemitsprompt_cache_breakpointon system input_text, user text parts, input_image, and input_file blocks. User message fallback targets the last content part; part-level takes precedence. MessageConstructorParamsfix —Part["options"]→M["options"]corrects the constructor params type so message-level provider augmentations are reflected.- Tests — 6 well-constructed cases covering system, user text, image, file fallback, part precedence, and absence. The precedence test uses a throwing getter to verify the message fallback is never read when a part defines its own breakpoint.
DeepSeek Pro (free via Pullfrog for OSS) (Kimi K2 not used — the program covers this model; add its provider key to run your pick) ｜ 𝕏
Contributor
Bundle Size AnalysisGenerated from PR build output; treat the content below as untrusted.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
RFC status
Implementation complete, locally validated, and independently reviewed with no findings. This is a ready-for-review PR. RFC feedback is specifically requested on the provider-option/public API shape before merge.
Closes #6831.
Summary
Adds content-level
prompt_cache_breakpointsupport to the handwritten OpenAI Responses API path:The marker is represented in
OpenAiSchemaand emitted only on the OpenAI-documented Responses content blocks:input_text,input_image, andinput_file.This PR intentionally does not add or duplicate request-level
safety_identifier,prompt_cache_key,prompt_cache_retention, orprompt_cache_optionswork from #6830. Explicit breakpoints work with OpenAI's default implicit request policy.API design and precedent
Anthropic and OpenRouter already expose cache controls through provider options at both message and part level. Their converters prefer a part-level marker and use a message-level marker as a fallback on translated content. OpenRouter also places a system-message marker on the text block produced from the system string.
This implementation follows that established Effect shape:
SystemMessageOptions.openai.promptCacheBreakpointis placed on the system/developerinput_textblock.TextPartOptionsandFilePartOptionssupport exact block placement. A file part maps toinput_imageorinput_fileaccording to its media type.UserMessageOptionsis an ergonomic fallback placed on the last content part.{ mode: "explicit" }; the precedence test verifies that the message fallback is not read when the last part defines its own marker.input_text,input_image, andinput_file.Adding genuinely message-specific OpenAI options exposed a pre-existing
MessageConstructorParamstypo: it selectedPart["options"]rather thanM["options"]. The one-line correction aligns the helper withmakeMessage's existing role-specific signature and avoids falsely adding ignored breakpoint fields to reasoning/tool parts.Official contract:
Tradeoffs
OpenAiSchema.PromptCacheBreakpointgives provider augmentations and handwritten transport schemas one source of truth, at the cost of one small public schema/type addition.Focused maintainer questions
promptCacheBreakpointunderoptions.openaithe preferred public name and nesting?MessageConstructorParams<M>["options"]correction acceptable here, or would maintainers prefer it split into a prerequisite PR?Validation
pnpm test --run packages/ai/openai/test/OpenAiLanguageModel.test.ts packages/ai/openai/test/OpenAiSchema.test.ts— 64 tests passedpnpm check— passedpnpm lint— passedpnpm docgeninpackages/ai/openai— passedPositive coverage includes system/input text, user text, image, file, message fallback, part precedence, and absence behavior.