Repository navigation
V67: early stopping for the boosting, measured and not built - #167
Merged
Merged
Conversation
V66 showed the boosting's validation error lowest well before its last round on 17 runs of 30. Before building early stopping, it was measured: six demos, ten splits each, a quarter of the rows kept away from everything, the 120 fixed rounds against a stop at the lowest validation error, a stop on rows held out inside the training rows, and that stop followed by a refit. On the metric the leaderboard ranks on, stopping early changes nothing on four demos, loses 4 accuracy points on reviews, and helps only on energy, where the boosting does no better than the mean. It improves the log loss, not the predictions. A stop chosen on the validation rows also flatters the boosting on the rows that choose the family. Nothing changes in training. The measure is recorded in gbdt.ts and PLAN.md, the glass box no longer suggests that fewer rounds would predict better, and the docs carry the table in both languages. F9 is closed. Version 1.67.0. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AyBsUet8WybTG986ubroar
dapiced
marked this pull request as ready for review
November 13, 2026 19:19
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
F9, opened by V66: the glass box had shown the boosting's validation error lowest well before its 120th round on 17 runs of 30. The obvious fix is early stopping. This wave measured the fix before writing it, and the measure says no. Nothing changes in training: no figure of any run moves and no golden reference is regenerated.
The trap seen before measuring
The validation rows choose the family. A boosting that also chooses its number of rounds on them would be scored on rows it tuned itself on. So three stops were compared against the 120 fixed rounds:
Measured
Six demos, ten splits each, a quarter of the rows kept away from everything. Stop A against 120 rounds, on those rows:
What ships
train/gbdt.ts, where the next reader will look for early stopping.Not in this wave
A stop tied to the ranking metric when the user ranks on log loss (the measure does favour fewer rounds there on two demos of four, but the validation trap has to be solved first); tuning depth or learning rate; the network or the forest.
Checks
xlsxtypes unreachable in my sandbox), format: clean.spreadsheet.test.ts(thexlsxtarball cannot be fetched in my sandbox; CI is the reference). No test added: no behaviour changed, one assertion follows the new sentence.lab.spec.ts:31, the Excel upload, same cause).Generated by Claude Code