Skip to content

fix(review): gate the zero-error label floor at the Bonferroni split delta - #9922

Merged
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
bitfathers94:fix/issue-9637
Jul 29, 2026
Merged

fix(review): gate the zero-error label floor at the Bonferroni split delta#9922
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
bitfathers94:fix/issue-9637

Conversation

@bitfathers94

Copy link
Copy Markdown
Contributor

fix(review): gate the zero-error label floor at the Bonferroni split delta

calibrateActThreshold compared the label count against a floor computed at
the raw delta, but the scan itself tests each candidate at delta/K after
#9066's Bonferroni split. A zero-error calibration set that clears the raw
floor but falls short of the split-delta floor fell through the whole scan
and was misreported as no_certifiable_threshold (an error-rate shortfall)
instead of insufficient_labels (a label-count shortfall), losing #9048's
status split. Also stop reporting a recalibration infra failure as
insufficient_labels in runRiskControlRecalibration's catch.

Closes #9637

…delta

calibrateActThreshold compared the label count against a floor computed at
the raw delta, but the scan itself tests each candidate at delta/K after
JSONbored#9066's Bonferroni split. A zero-error calibration set that clears the raw
floor but falls short of the split-delta floor fell through the whole scan
and was misreported as no_certifiable_threshold (an error-rate shortfall)
instead of insufficient_labels (a label-count shortfall), losing JSONbored#9048's
status split. Also stop reporting a recalibration infra failure as
insufficient_labels in runRiskControlRecalibration's catch.
@bitfathers94
bitfathers94 requested a review from JSONbored as a code owner July 29, 2026 23:14
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@loopover-orb

loopover-orb Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Tip

✅ LoopOver review result - approve/merge recommended

Review updated: 2026-07-29 23:24:38 UTC

4 files · 1 AI reviewer · no blockers · readiness 98/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
This PR closes a genuine, well-traced bug: calibrateActThreshold gated the zero-error label floor at the raw δ (`needed`) instead of the Bonferroni split-δ (`effectiveNeeded`) the scan's inner loop actually tests each candidate at, so a set that cleared the raw floor but not the split floor fell through to `no_certifiable_threshold` with a stale `bestUpperBound = Infinity` — exactly the failure the module's own comment warned was 'guaranteed never observed' under a guarantee that the split-delta fix now removes. I traced the math for all three new/updated regression tests (800/2-candidate, 59/59-candidate, 1000/1-candidate) and the new upfront `sorted.length < effectiveNeeded` early-return matches expectations exactly, and correctly prevents the loop from ever computing a bound with n below the split floor. The `runRiskControlRecalibration` catch-path change from `insufficient_labels` to a new `error` literal is a clean, well-motivated distinction (infra failure vs statistical shortfall) and is exercised by an updated test.

Nits — 4 non-blocking
  • src/review/risk-control-wire.ts: `runRiskControlRecalibration`'s return type widened to include `"error"` — verify no caller outside this diff pattern-matches exhaustively over `CalibrationResult["status"]` and would silently mishandle the new literal.
  • test/unit/risk-control-wire.test.ts: the extensive inline derivations (e.g. '736, not the raw 598') are helpful but duplicate the module's own `minimumCalibrationLabels` math in prose — a small risk of comment drift if the formula changes again.
  • Consider a short comment at the `effectiveNeeded` early-return in risk-control.ts noting it is the fix for the exact Infinity-bound failure mode the pre-fix comment above assumed was unreachable, to make the causal link even more explicit for future readers.
  • If any dashboard or alerting code consumes `runRiskControlRecalibration`'s summary object, grep for exhaustive switches over its status literals to confirm `"error"` doesn't fall through unhandled.

Decision drivers

  • ✅ Code review — No blockers (1 reviewer)
  • ✅ Gate result — Passing (No configured blocker found.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ✅ Linked #9637
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 111 registered-repo PR(s), 77 merged, 1 issue(s).
Contributor context ✅ Confirmed Gittensor contributor bitfathers94; Gittensor profile; 111 PR(s), 1 issue(s).
Improvement ✅ Minor risk: clean · value: minor · LLM: moderate
Linked issue satisfaction

Addressed
The diff computes effectiveNeeded = minimumCalibrationLabels(alpha, delta/candidates.length) and uses it both for the insufficient_labels gate and the per-candidate n>=needed check, adds the required regression tests (59/59 -> insufficient_labels with needed=138, and a no_certifiable_threshold case above the split floor), widens runRiskControlRecalibration's catch to return "error" with a covering

Review context
  • Author: bitfathers94
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: TypeScript
  • Official Gittensor activity: 111 PR(s), 1 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Keep the PR focused and include validation evidence before maintainer review.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 29, 2026
@codecov

codecov Bot commented Jul 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.56%. Comparing base (4b57927) to head (86407e9).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9922      +/-   ##
==========================================
+ Coverage   79.52%   79.56%   +0.03%     
==========================================
  Files         281      283       +2     
  Lines       58577    58699     +122     
  Branches     6833     6871      +38     
==========================================
+ Hits        46581    46701     +120     
  Misses      11706    11706              
- Partials      290      292       +2     
Flag Coverage Δ
backend 98.36% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/review/risk-control-wire.ts 100.00% <100.00%> (ø)
src/review/risk-control.ts 96.55% <100.00%> (ø)

@loopover-orb loopover-orb Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LoopOver approves — the gate is satisfied and CI is green.

@loopover-orb
loopover-orb Bot merged commit 45471e7 into JSONbored:main Jul 29, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

orb(risk-control): the zero-error label floor ignores the Bonferroni δ/K split

1 participant