Prompt caching drops to 0% when introducing base64 file inputs

[shahin43]

Hi team,

we are testing prompt caching behaviour using Agents SDK and noticed an unexpected drop in cache (0%) when introducing file inputs (as base64 encoded)

---

Setup

• Model: gpt-5.4-mini
• SDK: Python (custom script)
• File input: PDF via base64 (input_file)
• Conversation store enabled
• Prompt cache key added
• Response chaining enabled using previous_response_id
• Prompt:
  • Using Chat Prompts; passing prompt_id as instructions
  • Large static prefix (~13k tokens)
  • Same system instructions
  • Same tool definitions
  • Only user message varies

---

Results

Round 1: text_only_first     cache_hit=96.7%
Round 2: text_only_repeat    cache_hit=96.7%

Round 3: file_input_first    cache_hit=0.0%   <-- unexpected (even though considerable cache drop expected due to increased input tokens)
Round 4: text_only_new       cache_hit=96.7%

Round 5: file_input_repeat   cache_hit=90.3%
Round 6: text_only_new       cache_hit=96.7%

We tried this few times and above behaviour is consistent across Agents SDK and using Responses API directly

---

Key observation

• Adding a file input causes cache to drop to 0%, even though:

  • Prompt prefix is unchanged
  • Tool definitions are unchanged

• Repeating the same file input restores cache (~90%)

• Tried different options, using a prompt_cache_key, for better caching rates but observing same issue and it has got zero impact on this

---

Expected behaviour

• Cache % to drop due to additional tokens or input from files
• But not drop to 0%, since the shared prefix, instructions, tools are still same

---

Questions

1. Any known issues with input files and cache hits, file inputs treated as part of the cacheable prefix in a way that invalidates prior cache ?
2. Does base64 encoding prevent prefix matching across requests ?
3. Would using file_id instead of base64 improve cache reuse?
4. Is this expected behaviour or a potential issue ?

---

Hypothesis

It seems that introducing file inputs:

• Changes the effective prompt prefix completely
• Causes cache routing to treat the request as a new prefix

---

Additional notes

• All prompts exceed the 1024 token threshold
• Cache works consistently for text-only flows
• Issue appears only when mixing text-only and file-input requests

---

Would appreciate any clarification or best practices for maintaining cache efficiency in mixed input scenarios.

Happy to provide a minimal repro script if needed.

[seratch]

Thanks for writing in. If you have repro steps, sharing it with us would be appreciated.

[JiwaniZakir]

The 0% on first file introduction followed by ~90% on repeat is the telling pattern here — it indicates the cache entry simply didn't exist yet for that token sequence, not that caching is broken. When a base64-encoded file is inlined, its content is tokenized and inserted into the message sequence, which shifts all subsequent tokens and creates a completely new cache key that has never been seen before, explaining the cold-start 0% miss. The prompt_cache_key hint has no effect here because OpenAI's prefix caching is based on exact token-sequence matching, not logical content identity, so even an identical static prefix becomes a cache miss the moment the downstream token stream differs from what was previously cached. Using file_id instead of raw base64 is very likely to improve this — a file reference is a compact, stable token sequence regardless of file size, which means the surrounding prefix structure remains consistent across requests and gives the caching layer a stable prefix to match against.