Skip to content

fix(gateway): prevent body budget starvation (Core 0.98.1) - #407

Merged
fylorn merged 1 commit into
mainfrom
codex/fix-body-budget-starvation
Nov 17, 2026
Merged

fylorn merged 1 commit into
mainfrom
codex/fix-body-budget-starvation

Conversation

@fylorn

@fylorn fylorn commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

Large bodies can repeatedly receive Core's memory-budget 429 while later small requests continue to enter. This changes admission to wait in FIFO order for space, with a 30-second bound, so an existing large waiter cannot be overtaken by new small bodies.

Known-size uncompressed uploads reserve their complete bounded size before reading and remain concurrent. Only one unknown-size or compressed body can grow while holding a partial reservation, avoiding partial-body deadlocks. When decompression exhausts the budget, it releases the failed output and CPU slot, waits for exclusive space, and retries at most once. Completed decompression returns the compressed-input headroom; cancellation, idle uploads and timeouts release their reservations. Budget exhaustion still returns the existing retryable 429 and now logs occupancy.

Bumps Core to 0.98.1 and documents the admission behavior and release. Addresses #403's reproduced starvation mechanism; the original deployment's response body, version and budget occupancy remain unavailable, so the field incident's cause is not asserted as confirmed.

Validation:

  • cargo fmt --all -- --check and cargo clippy --workspace --all-targets -- -D warnings passed.
  • cargo nextest run --workspace --profile ci: 4510 passed, 16 skipped; workspace doc tests passed.
  • scripts/smoke.sh: 87 passed, 0 failed; release-note tests: 17 passed.
  • Regression coverage includes FIFO large/small admission, chunked growth, cancelled waiters, bounded waits, idle uploads, decoded headroom, and real HTTP recovery with parallel known-size uploads.

Immediate budget refusal lets small requests keep overtaking a large
conversation on every retry. Queue admission before reading, reserving a
known uncompressed upper bound in advance while serializing growth of
unknown and compressed bodies to avoid partial-reservation deadlocks.

Bound waits and idle uploads at 30 seconds, release CPU slots while waiting
for decompression space, and return compressed-input headroom promptly.
Retain the existing memory policy, size limits and retryable 429 contract.

Release Core 0.98.1 with regression coverage and updated server docs.
Addresses #403; the original field incident still lacks runtime evidence.
@fylorn
fylorn merged commit df1022c into main Nov 17, 2026
7 checks passed
@fylorn
fylorn deleted the codex/fix-body-budget-starvation branch November 17, 2026 03:41
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 17, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant