[aywrite]

scripts/fit_attention.py split train and holdout by fen. Rows from one root share a parent node and most of a feature vector, so that split can put a row and its sibling on opposite sides. This adds what is needed to check whether that matters.

• --group-by FILE names the group each ledger's rows belong to and splits by group. Record the ledger one root per file and the map attributes every row to its root.
• --drop FEATURE leaves a feature out of the fit and writes a zero for its constant.
• The held out auc is printed over every row, over depth 4 and up (where the gate acts) and per depth, with a bootstrap standard error that resamples the split's own unit: the group, or the fen without a map.

Nothing in the engine changes. No weight, no threshold.

What it showed, on a depth 8 ledger over 1,428 roots from our own games (13.7 million scouts, fitted on a 20% row sample, grouped by source game):

	depth 4+ holdout auc
split by game	0.7674 ± 0.0093
split by fen	0.7723 ± 0.0038
hist_milli dropped, by game	0.7652 ± 0.0092
alpha_gap dropped, by game	0.7453 ± 0.0100

On a corpus this wide the fen split does not inflate the auc. It understates its error by about 2.4 times. hist_milli can go at no measured cost and alpha_gap cannot.

Tests: 26 in test_fit_attention.py, the whole script suite, and both cargo test runs pass. A second agent reviewed the change; its four minor findings are folded in.

🤖 Generated with Claude Code

[github-actions]

Speed against 9e4bb4c3

Both sides built and run on this runner in this job, the way
scripts/speed.sh measures a perf commit. The change is between
medians, and the spread beside it is what the change has to be
read against. The node counts are the search's: they move when
the search does, and a speed change leaves them alone.

round     base nps  candidate nps
    1     10589896       10691471
    2     10952794       10977811
    3     10887452       10792514
    4     10801403       10997032
    5     10953538       10959061
    6     11026744       10957540
    7     10975444       10954686
    8     10949167       10975538
    9     10895393       10518610

             nodes    time  median nps  fastest nps
base       3868987  0.35 s    10949167     11026744
candidate  3868987  0.35 s    10957540     10997032
change                           +0.1%        -0.3%

the change is inside the spread, so the medians make no claim; the
fastest column is the steadier comparison when the machine was not
quiet

Speed: +0.1% (bench nps, 9 interleaved rounds vs 9e4bb4c3, spread 4.4%)