Pre-submission checklist
Bug Description
Summary
core/memory/l2/gain.ts's adaptiveBaseline() floors at MIN_ADAPTIVE_BASELINE = 0.2. After the v2.0.7 reward-backprop rewrite (normalized credit assignment instead of right-to-left decay), typical pool-mean trace values dropped roughly an order of magnitude (from the old ~0.5–0.85 range down to ~0.01–0.06). Since adaptiveBaseline(poolMean) = max(MIN_ADAPTIVE_BASELINE, min(V7_NEUTRAL_BASELINE, poolMean)), at these post-rewrite pool means the function always returns the 0.2 floor — it's dead code that never adapts to the actual reward distribution anymore. Two concrete consequences:
- Gain ceiling:
computeGain() (effectiveWith − shrinkTowardBaseline(...)) is capped well below what candidate policies can realistically reach, since effectiveWith values now cluster near 0.01–0.06 while shrinkTowardBaseline pulls toward the stuck 0.2 baseline.
- Negative-gain skew for low-
without-count policies: shrinkTowardBaseline uses WITHOUT_PRIOR_PSEUDOCOUNT = 5 to blend toward the (now systematically too-high) 0.2 baseline when a policy has few "without" observations — this drives gain strongly negative for exactly the policies that most need the promotion gate to evaluate them fairly.
Net effect: candidate policies (and, via the shared algorithm.skill.minGain/minSupport config block, candidate skills too) can sit indefinitely below the promotion threshold even when they are performing well by any reasonable read of the actual v2.0.7 reward distribution.
Root cause
// core/memory/l2/gain.ts
export const V7_NEUTRAL_BASELINE = 0.5;
export const MIN_ADAPTIVE_BASELINE = 0.2;
export function adaptiveBaseline(poolMean: number): number {
if (!Number.isFinite(poolMean)) return V7_NEUTRAL_BASELINE;
return Math.max(MIN_ADAPTIVE_BASELINE, Math.min(V7_NEUTRAL_BASELINE, poolMean));
}At poolMean ≈ 0.01–0.06 (the v2.0.7-typical range), Math.min(0.5, poolMean) ≈ poolMean, and Math.max(0.2, poolMean) always resolves to 0.2 since poolMean < 0.2. The constants and the doc comments around them were written for the pre-v2.0.7 reward scale and were never revisited when core/reward/backprop.ts was rewritten.
Impact
- Policy/skill promotion effectively stalls fleet-wide post-v2.0.7 unless
minGain is tuned drastically lower than its documented/default value — and even then, the baseline floor keeps distorting relative gain across policies with different without-support counts.
core/config/defaults.ts has comments describing thresholds like minTraceValue: 0.005 in terms of the new reward scale ("Reward backprop V values for typical multi-step turns are clustered around 0.02–0.5"), showing the intent to retune for v2.0.7 was there for some settings but missed gain.ts's baseline constants.
Suggested Fix
MIN_ADAPTIVE_BASELINE (and possibly V7_NEUTRAL_BASELINE) need to be re-derived against the v2.0.7 reward-value distribution rather than left at their pre-rewrite values — e.g. computed as a percentile of the live pool distribution rather than a fixed constant, or at minimum lowered to match the new empirical range. Happy to contribute data/a PR once maintainers confirm the intended approach (fixed retuned constants vs. a genuinely adaptive percentile-based baseline).
Pre-submission checklist
Bug Description
Summary
core/memory/l2/gain.ts'sadaptiveBaseline()floors atMIN_ADAPTIVE_BASELINE = 0.2. After the v2.0.7 reward-backprop rewrite (normalized credit assignment instead of right-to-left decay), typical pool-mean trace values dropped roughly an order of magnitude (from the old ~0.5–0.85 range down to ~0.01–0.06). SinceadaptiveBaseline(poolMean) = max(MIN_ADAPTIVE_BASELINE, min(V7_NEUTRAL_BASELINE, poolMean)), at these post-rewrite pool means the function always returns the0.2floor — it's dead code that never adapts to the actual reward distribution anymore. Two concrete consequences:computeGain()(effectiveWith − shrinkTowardBaseline(...)) is capped well below what candidate policies can realistically reach, sinceeffectiveWithvalues now cluster near 0.01–0.06 whileshrinkTowardBaselinepulls toward the stuck 0.2 baseline.without-count policies:shrinkTowardBaselineusesWITHOUT_PRIOR_PSEUDOCOUNT = 5to blend toward the (now systematically too-high) 0.2 baseline when a policy has few "without" observations — this drives gain strongly negative for exactly the policies that most need the promotion gate to evaluate them fairly.Net effect: candidate policies (and, via the shared
algorithm.skill.minGain/minSupportconfig block, candidate skills too) can sit indefinitely below the promotion threshold even when they are performing well by any reasonable read of the actual v2.0.7 reward distribution.Root cause
At
poolMean≈ 0.01–0.06 (the v2.0.7-typical range),Math.min(0.5, poolMean)≈poolMean, andMath.max(0.2, poolMean)always resolves to0.2sincepoolMean < 0.2. The constants and the doc comments around them were written for the pre-v2.0.7 reward scale and were never revisited whencore/reward/backprop.tswas rewritten.Impact
minGainis tuned drastically lower than its documented/default value — and even then, the baseline floor keeps distorting relative gain across policies with differentwithout-support counts.core/config/defaults.tshas comments describing thresholds likeminTraceValue: 0.005in terms of the new reward scale ("Reward backprop V values for typical multi-step turns are clustered around 0.02–0.5"), showing the intent to retune for v2.0.7 was there for some settings but missedgain.ts's baseline constants.Suggested Fix
MIN_ADAPTIVE_BASELINE(and possiblyV7_NEUTRAL_BASELINE) need to be re-derived against the v2.0.7 reward-value distribution rather than left at their pre-rewrite values — e.g. computed as a percentile of the live pool distribution rather than a fixed constant, or at minimum lowered to match the new empirical range. Happy to contribute data/a PR once maintainers confirm the intended approach (fixed retuned constants vs. a genuinely adaptive percentile-based baseline).