Skip to content

perf(eval): Cache the mobility fold by the position key (measured, not for merge) - #250

Closed
aywrite wants to merge 1 commit into
masterfrom
eval/mobility-cache
Closed

aywrite wants to merge 1 commit into
masterfrom
eval/mobility-cache

Conversation

@aywrite

@aywrite aywrite commented Sep 13, 2026

Copy link
Copy Markdown
Owner

Not for merge. The cache works, saves instructions and costs time. This is
the measurement, opened where it can be read rather than left on a branch.

specs/mobility-cost.md said a position keyed cache could not work, because
mobility_counts reads the full occupancy and the occupancy changes every
move. The hit rate says otherwise. Over the bench, 29.21% of the term's
2,003,154 calls are on a position already scored in that search, and 18.43%
land on a live entry of a table of 16,384. A transposition entry cuts only
when the depth it was stored at suffices, so a position reached again deeper
is searched again and evaluated again, and iterative deepening walks the same
tree up to seven times over. The table removes the search below a repeat, not
the evaluation at it.

So it was built.

What it saves

At the same 3,868,987 nodes the run goes 4,030,086,289 instructions to
3,966,023,843, -1.59%. The term goes 13.11% of the run to 11.91%: its miss
path falls 18.0%, which matches the hit rate, and the probe costs 40,689,060
back.

What it costs

The clock does not follow. At depth 9, fastest round of each side against the
same base binary:

side readings mean range
the cache 6 -0.92% -2.3% to +0.1%
a null, same md5 both sides 5 +0.06% -0.3% to +0.8%

The difference is -0.98% (+/- 0.44), two standard errors putting it
between -1.85% and -0.10%, which does not reach zero. It executes 1.59% fewer
instructions and takes about 1% longer.

Callgrind offers no reason: last level misses are 295,846 against the bypassed
tree's 295,843, flat to three digits. It models a first level and a last
level, and neither an L2 nor a TLB, which is what a 256KB table probed at
random pays. performance.md already records that 96% of this engine's L1
misses are served by L2.

The null test, which is the part worth keeping

Every clock reading taken on this branch before the null was noise. A sweep of
five table sizes read -0.2%, +0.5%, -5.3%, -6.1% and -1.7%, which looks like a
memory effect and is five samples of the machine. Two binaries built from the
same commit, the same md5, compared over 25 interleaved rounds at the default
depth, read +1.1% between the medians and -2.5% between the fastest
rounds
.

docs/DEVELOPMENT.md says the fastest pair is the one that survives a loaded
runner, since nothing sharing the machine makes a run faster. That holds once
a round is long enough to swamp what the clock does at a process start. The
default round is 0.34s. Depth 9 is 4.4s and closes the null band to +0.8% and
-0.3%, which is what let a 1% effect be read at all. The Speed: trailer's
grammar has no field for the depth, so a reading at 0.34s and one at 4.4s are
written down the same way.

What is left, both the owner's call

  • An eight byte entry. A 32 bit fragment of the key beside the score
    halves the footprint at the same slot count, which is the only variant the
    evidence points at. Callgrind cannot rule on it, having already reported no
    misses to remove, so it costs an hour of clock readings to decide.
  • The ceiling. Mobility is 11.91% of the run with this and 13.11% without
    it, against the 5% specs/mobility.md allows a leaf term, and the refit
    that put it there is worth +76 elo. Either the figure means something, in
    which case the term needs incremental maintenance, or it is a rule of thumb
    and should say so.

One thing worth carrying into either: what repeats is the position, not the
mobility term.
The same table, keyed the same way, would answer the whole
evaluation rather than one term of it, against the same memory traffic that
sank this one. That is a different arm.

Notes

  • Based on 7e6ddd7, where every number above was measured. master has moved
    seven commits since, two of them touching comments in eval.rs.
  • eval/term-modules splits eval.rs into a module per term. If that lands
    first, this branch does not apply as it stands.
  • Re-verified on this tree today: 596 tests pass in the debug run, 459 plus
    the tactical suite's three in the release run, fmt clean, clippy quiet apart
    from the standing MAGIC const evaluation warning, and the bench reads
    3,868,987, which is the trailer.

🤖 Generated with Claude Code

`specs/mobility-cost.md` section 2 said a position keyed cache could not
work, because the full occupancy is among the term's inputs and moves every
ply. The hit rate says otherwise. The clock says the spec reached the right
conclusion by the wrong route.

**This does not ship.** It is kept as the measurement.

## What repeats

Over the bench, 29.21% of the term's 2,003,154 calls are on a position it has
already scored in that search, and 18.43% land on a live entry of a table of
16,384. A transposition entry cuts only when the depth it was stored at
suffices, so a position reached again deeper is searched again and evaluated
again, and iterative deepening walks the same tree up to seven times over. The
table removes the search below a repeat, not the evaluation at it. Section 5
of that spec said this finding falsifies section 2, and it does.

## What it saves, and what it costs

Instructions fall 4,030,086,289 to 3,966,023,843, **-1.59%**, at the same
3,868,987 nodes. The term goes 13.11% of the run to 11.91%: the miss path
falls 18.0%, which matches the hit rate, and the probe costs 40,689,060 back.

The clock does not follow. At depth 9, fastest round of each side, six
readings of the cache against the same base binary come to **-0.98%
(+/- 0.44)** where five readings of a null come to +0.06%. Two standard errors
put the difference between -1.85% and -0.10%, which does not reach zero. It
executes 1.59% fewer instructions and takes about 1% longer.

Callgrind offers no reason for it: last level misses are 295,846 against the
bypassed tree's 295,843, flat to three digits. It models a first level and a
last level, and neither an L2 nor a TLB, which is what a 256KB table probed at
random pays. The saving is real and the payment is in a currency the profiler
does not carry.

## The null test, which is the part worth keeping

Every clock reading taken on this branch before that null was noise. A sweep
of five table sizes read -0.2%, +0.5%, -5.3%, -6.1% and -1.7%, which looks
like a memory effect and is not one. Two binaries of the same md5, compared by
`scripts/speed.py` over 25 interleaved rounds at the default depth, read +1.1%
between the medians and **-2.5% between the fastest rounds**.

`docs/DEVELOPMENT.md` says the fastest pair is the one that survives a loaded
runner, since nothing sharing the machine makes a run faster. That holds once
a round is long enough to swamp what the clock does at a process start. The
default depth is 0.34s a round and its null band is 2.5% wide. Depth 9 is 4.4s
a round and closes it to +0.8% and -0.3% over five readings. **A one percent
claim needs a round measured in seconds**, and the trailer below cannot say
which was used: its grammar has no field for the depth.

## What is left

One variant fits the evidence: an eight byte entry, a 32 bit fragment of the
key beside the score, which halves the footprint at the same slot count.
Callgrind cannot rule on it, having already reported no misses to remove, so
it costs an hour of clock readings to decide. That is the owner's call rather
than this branch's.

Bench: 3868987
Speed: -0.6% (bench nps, 15 interleaved rounds vs 7e6ddd7, spread 33.5%)
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Speed against 0b7c6245

Both sides built and run on this runner in this job, the way
scripts/speed.sh measures a perf commit. The change is between
medians, and the spread beside it is what the change has to be
read against. The node counts are the search's: they move when
the search does, and a speed change leaves them alone.

round     base nps  candidate nps
    1      9861286        9885777
    2      9935050        9359327
    3      9641109        8766692
    4     10483979       10198264
    5      8875187        9115080
    6      9684185        9578383
    7      9889492       10491087
    8      9176435       10081919
    9      9101742        9518599

             nodes    time  median nps  fastest nps
base       3868987  0.40 s     9684185     10483979
candidate  3868987  0.40 s     9578383     10491087
change                           -1.1%        +0.1%

the change is inside the spread, so the medians make no claim; the
fastest column is the steadier comparison when the machine was not
quiet

Speed: -1.1% (bench nps, 9 interleaved rounds vs 0b7c6245, spread 18.0%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant