codex-516-hook — warn when gpt-5.5 truncates its reasoning in Codex

gpt-5.5 serving Codex sometimes stops its hidden reasoning at exactly 516 output tokens and answers anyway, in its normal confident register (openai/codex#30364, HN discussion) — under both ChatGPT-subscription and API-key auth. In our measurements, truncated turns were wrong on many nontrivial reasoning tasks, while every untruncated run of the same prompts was correct.

This hook reads the per-turn reasoning_output_tokens that Codex already writes to the local session transcript, and when a turn hits exactly 516:

• at turn end (Stop hook): a [c516] warning renders in the Codex TUI, so you know the answer above it is suspect before you act on it;
• on your next message (UserPromptSubmit hook): a warning is injected into the model's context telling it the previous turn was truncated, not to build on that turn's conclusions, to redo the thinking, and to tell you it did so.

If you want the truncation automatically repaired rather than flagged, see CodexCont and codexcomp: they run a local proxy that catches the truncation in-flight and auto-continues the reasoning. This hook is the lighter-touch complement — it leaves Codex's connection to the API untouched and simply flags truncations after the fact, to you and to the next turn's model.

Install

Requirements: Python 3.8+ on PATH (stdlib only). We've tested on codex-cli 0.142.3 on Windows only.

1. Clone this repo anywhere; on macOS/Linux chmod +x c516-hook.sh. The shims locate hook.py relative to themselves, so keep the files together.
2. Add to ~/.codex/config.toml, using the absolute path of your clone. command is what macOS/Linux runs; command_windows overrides it on Windows — set whichever applies (or both):

[[hooks.UserPromptSubmit]]
[[hooks.UserPromptSubmit.hooks]]
type = "command"
command = '/path/to/clone/c516-hook.sh'
command_windows = 'C:\path\to\clone\c516-hook.cmd'

[[hooks.Stop]]
[[hooks.Stop.hooks]]
type = "command"
command = '/path/to/clone/c516-hook.sh stop'
command_windows = 'C:\path\to\clone\c516-hook.cmd stop'

3. Start Codex interactively and approve the one-time trust prompts for the two hook registrations (/hooks shows their status). Until trusted, Codex silently skips the hooks; sessions already open when you edited the config don't load them either — use a fresh session.

Testing that it works

The puzzles/ directory contains two prompts with machine-verified unique answers, tuned against this bug.

1. puzzles/clamp-trigger.txt — makes the warning fire (correct answer: 24)

An easy-looking drawing puzzle that triggered the truncation in ~90% of our runs (gpt-5.5, reasoning effort xhigh). It is deliberately easy enough that the model usually still answers 24 correctly even when truncated — its job is to fire the hook reliably, not to produce wrong answers.

Expected behavior in a fresh interactive session:

1. Paste the puzzle. The turn ends fast (~20 s; a healthy run takes longer and visibly thinks). If it truncated, the [c516] Stop warning appears in the TUI right after the answer: "This turn's reasoning was truncated at exactly 516 tokens…".
2. Send any follow-up message in the same session ("thanks" is enough). The reply should begin by telling you the truncation warning fired, and should re-derive the answer rather than repeating it. (In our testing the redo turn is often truncated again — on this easy puzzle it still lands on 24. That is the bug, not the hook.)
3. No warning? Check the log (below). If the previous turn's count isn't 516, the run escaped the bug (~10% on this puzzle) — retry in a fresh session.

Note the submit-side warning only fires on a second message in the same session — if you re-roll the puzzle in fresh one-shot sessions you will only ever see the Stop warning.

2. puzzles/wrong-answer-zebra.txt — demonstrates the damage (correct answer: house 2)

A zebra-style logic puzzle (unique solution, verified by exhaustive search; the fish is never mentioned in the clues, so the answer requires solving the whole grid). It truncates in only ~15% of runs, but truncated runs answered wrong in 5 of our 6 observed truncations (scattered guesses: 1, 4, 5) — with a confident-sounding justification. Healthy runs think for a minute or more and consistently answer house 2.

Re-roll it in fresh sessions until one answers suspiciously fast: that run should combine a wrong answer with the [c516] TUI warning — the full failure mode, caught in the act.

The log

Every prompt submission and turn end is logged (default ~/.local/state/c516-hook.log, override with C516_HOOK_LOG), one JSON line per invocation, including the previous turn's full reasoning-count list — useful both for verifying the hook and for passively monitoring how often the bug hits your real sessions:

{"now": "...", "mode": "stop", "counts": [516], "truncated": [516], "emitted": true, ...}

To smoke-test the plumbing without the bug: C516_FORCE_EMIT=1 forces the warning on the next invocation.

Expected behavior summary

Situation	Stop hook	Submit hook
Previous turn had a response at exactly 516 reasoning tokens	TUI warning at turn end	Instance warning on next same-session message
Healthy turn (any other counts, incl. 1034/2070/2588/3106)	silent	silent
First message of a session / no previous turn	silent	silent
Transcript missing, unreadable, or format changed	silent (fail-open, logged)	silent (fail-open, logged)

Counts at 1034/1552/2070/2588/3106… (516+518k) are common in healthy runs — in our data every such run answered correctly (it appears to be count quantization, not harmful truncation), so the hook deliberately ignores them. If the serving behavior changes, adjust TRUNCATION_VALUES in hook.py.

What triggers it: measurements (2026-07-05)

The truncation follows Codex's request shape, not the account type. Same trigger prompt (puzzles/clamp-trigger.txt), gpt-5.5, reasoning effort xhigh, codex-cli 0.142.3:

Request	Runs	exactly 516 reasoning tokens
codex exec, ChatGPT auth	20	18
codex exec, API-key auth (same context)	20	18
bare /v1/responses, API key, same prompt, no tools	20	0

Exactly what in Codex's request shape triggers it is unknown; a quick check hinted that tool declarations matter and input size alone doesn't, but we didn't chase it further.

Reasoning counts on every path land disproportionately on the 516+518k ladder (1034, 1552, 2070, 2588, ...) — the quantization comb shows even in healthy runs; the harmful event is specifically termination at the first rung.

Design note: mid-turn vs final-response truncation

A turn can contain many model responses (each tool-call round is one), and the hook fires when any of them hit exactly 516. In our own session history the hits split roughly 55% mid-turn-only / 40% final-response / 5% single-response turns. Those differ in severity: mid-turn truncation clips intermediate steps (what to read, what to explore next) — real but diffuse damage — while final-response truncation clips the response that writes the actual answer, which is where the wrong-answer correlation was measured.

The warning text is calibrated to the final-response case, so when it fires treat it as an upper bound on alarm: the log line's counts list shows where the 516s sat (last element = the answer-writing response). A natural refinement would be to soften the injected message when the hits were mid-turn only; we haven't shipped that because we haven't yet tested it in live use.

Caveats

• Not an account-type issue: every Codex user is exposed regardless of how they pay (see the measurements section above).
• A turn whose reasoning naturally lands on exactly 516 would false-positive — and from where this hook sits it would be indistinguishable from the bug, so we can't count how often that happened to us. Two indirect reasons to think it's rare: healthy runs landed near 516 without hitting it (503, 533, 554, 595), and the issue's aggregate data puts non-gpt-5.5 models' exact-516 rate at ~1.3%. But on long terms the probability increases - we haven't done enough testing here yet.
• Maturity: this works on the author's setup (Windows 11, codex-cli 0.142.3, ChatGPT Pro, Python 3.12), verified end-to-end there, but it has not been tested much beyond that — treat it as a working example rather than hardened tooling. The rollout format is platform-independent, but the .sh shim in particular is untested. Reports welcome.
• The hook reads only the local session transcript; nothing is sent anywhere.

License

MIT