Skip to content

gate(mutation): ratchet graduates on stability alone — a 28.75% score can lock in as the permanently gated floor #1457

Description

@thymikee

Context

The mutation gate's graduation logic in scripts/mutation/ratchet.ts promotes a decision kernel from advisory to gating when its score is stable for requiredStableRuns: 2 consecutive weekly sweeps (mutation-baselines/decision-kernels.json, mutation-weekly.yml, mutation-affected.yml; umbrella #1412 / gate issue #1415). There is no minimum-score condition. Verified consequence (2026-07-28 audit, adversarially confirmed): scroll-edge-state sits at 28.75% (228 survived mutants) — if that number is merely stable twice, it graduates and the ratchet then enforces 28.75% as the protected floor forever. Stability of a bad score is not quality; the gate would begin blocking unrelated PRs to defend a floor that detects almost nothing.

Companion issues #1455 / (kernel-errors) raise the actual scores; this issue fixes the policy so graduation can't outrun them.

Proposed change (needs maintainer sign-off on the threshold)

Add a minimum-score condition to graduation: a kernel becomes gating only when score >= FLOOR (proposal: 70%) AND stability is met. Below-floor kernels stay advisory and are reported (in the weekly summary output) as "stable but below graduation floor".

Acceptance criteria

  • ratchet.ts graduation requires both conditions; floor is a named constant in the baseline JSON schema (per-kernel override allowed, default 70) so raising/lowering it is a data change with an audit trail, not code.
  • Unit tests in scripts/mutation/*.test.ts (the existing mutation:test lane) cover: below-floor+stable stays advisory; at-floor+stable graduates; already-graduated kernels are unaffected by the new rule (no regression of existing gating).
  • Weekly summary output distinguishes "advisory: unstable" from "advisory: below floor" so the state is visible, not silent.
  • Docs touched: docs/agents/testing.md section describing graduation criteria updated in the same PR.
  • pnpm mutation:test, pnpm check:tooling green. No baseline regeneration needed (policy change only).

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions