Do not add the CodeRabbit status to required checks: it reports success while throttled #85

Captured before branch protection is added, because this failure mode is invisible at the moment it is introduced — the checkbox goes green either way.

The hazard

CodeRabbit publishes a commit status under context CodeRabbit. When it is rate-limited it reports:

state:       success
description: Review rate limited

The state field reads success for a review that never ran. The truth lives only in the description.

Observed simultaneously across four PRs on 2026-07-27, during a repo-wide adaptive rate limit:

Why it matters for branch protection

main currently has no branch protection (GET /branches/main/protection → 404), so nothing consumes this status today and the vector is latent rather than live.

The trap is what happens when protection is added. Putting CodeRabbit in the required-status-checks list looks like a tightening and functions as a loosening: the rule would be satisfied by reviews that never happened, silently and machine-readably, with no signal that anything was skipped.

Recommendation

• Do not add CodeRabbit to required status checks.
• If CodeRabbit is to gate merges, enforce it with something that reads the description and rejects rate limited / paused / limit reached, not the state.

Related false-clean vectors on the same bot

A throttled CodeRabbit is indistinguishable from a clean one in at least four independent ways. All four were hit in a single session:

1. Zero unresolved review threads — identical whether CR reviewed and found nothing, or never reviewed. chore(make): add docker-skip convenience target to Makefile.terraform cloud-commitments-cli#1519 had five CR comments, all limit/pause notices, and zero threads.
2. Edit-in-place walkthrough — CR rewrites its summary comment, so one comment can carry a stale substantive walkthrough and a fresh rate-limit warning at once. On refactor(common): make DatabaseDetails/CacheDetails pointer-only cloud-commitments-cli#1525 the walkthrough created 12:39:19Z still carried the rate-limit marker after being edited at 17:38:04Z.
3. Green commit status — this issue.
4. Empty-bodied reviews — CR posts review objects with empty bodies (inline replies, not passes). A check for "a review exists since HEAD" counts these; a check for substantive content does not.

The only reliable signal is a CodeRabbit response newer than the head commit, carrying substantive content, with no throttle/pause marker among anything it posted since that commit.

Context

Found while driving nine open PRs to merge-ready state. Had merges been gated on thread count alone, seven unreviewed PRs would have merged clean — including a 7,700-line money path that was concurrently found to contain a double-spend across three clouds.