Isolate upstream #30270: correct fix, no measurable effect here, and a guard proven red

[bernardladenthin]

ggml-org/llama.cpp#30270 was merged with — in the
author's own words — "a better by-eyeball measurement" on a FreeBSD box with mismatched RAM
clocks. No numbers, no review comments. This is what a quantified answer looks like.

The method: one machine, one tree copied twice, one variable

llama.cpp b11574 cloned once and copied; the second copy had exactly the seven lines removed, with
the removal printed into the run log. Both built on an m8i.16xlarge (Xeon 6975P-C, 64 vCPU).
Numbers from upstream's own llama-bench, 5 repetitions, one module staged at a time — so each
figure carries a spread and our JNI layer is not in the picture at all.

The variant tiers come from upstream's own ggml/src/CMakeLists.txt, so every row carries a
prediction:

module	flags	prediction
sse42, sandybridge	no F16C	must not move — the control
ivybridge, piledriver	F16C, no AVX2	should gain
haswell, alderlake	F16C + AVX2	the case the PR names
cascadelake, sapphirerapids	+ AVX512	should gain less

The result is negative, and that is the finding

module	Δ pp512	Δ tg128
sse42 (control)	−0.4 %	−0.7 %
sandybridge (control)	+0.3 %	+0.5 %
ivybridge	−0.3 %	+0.5 %
piledriver	−0.1 %	−0.8 %
haswell	+1.8 %	+0.2 %
alderlake	−0.2 %	+1.5 %
cascadelake	+0.6 %	−0.9 %
sapphirerapids	+0.1 %	+1.2 %

Everything inside ±2 % with random signs. The controls — which the fix cannot touch — move as
much as the test cases, and haswell is +1.8 % at 16 threads but −0.8 % at 8.

The fix is still right: the #elif defined(__F16C__) branch defined only the COMPUTE macro, so
the selected one fell through to a 64K lookup table and the hardware instruction was never used. Its
gain is machine- and workload-dependent — the fix removes a ~256 KB table from the path, which costs
on a bandwidth-starved machine and nothing on a server CPU with a 4-bit quant.

It refutes something this repo wrote an hour earlier

A haswell jump from 221.8 to 422.7 t/s between a b11538 4-vCPU run and a b11558 64-vCPU one had been
read here as consistent with #30270, since haswell gained far more than the AMX module. Wrong:
#30270 contributes ~0, so that jump was the instance size. The correction is kept in
cpu-backend-modules.md rather than quietly removed — the next person comparing two tables across a
pin and a machine size will be tempted the same way.

A guard, proven both ways

#if defined(GGML_CPU_COMPUTE_FP16_TO_FP32) && !defined(GGML_CPU_FP16_TO_FP32)
#error "a branch defined GGML_CPU_COMPUTE_FP16_TO_FP32 but not GGML_CPU_FP16_TO_FP32"
#endif

clang -fsyntax-only against the real b11574 header: passes on the fixed tree, fails with the
#error on the reverted one, does not fire without __F16C__, and both trees compile when the
guard is absent — so the guard is the only thing that changes the outcome.

My first attempt at that test proved nothing: it appended the guard after the include, and the
header's own fallback always defines the macro, so it reported exit=0 for both trees. A guard that
passes in the broken case has to be suspected of being in the wrong place. Mistake and fix are both
recorded.

Also in here

cpu-backend-modules.md gains the b11558 tables from the 64-core machine as an additional
section — the b11538 ones are untouched, as that file's house rules require. They answer the open
high-core-count question: at 64 threads haswell leads generation at 192.6 while sapphirerapids
is last at 181.8 — and ggml picks the latter. That sharpens the existing module-scoring finding
rather than adding a new one.

TODO.md: five rows of the measurement table struck through, leaving only the two that need
hardware nobody rents. And the upstream section gains a fifth item that is a small PR rather than an
issue — move the two #defines into the branch they complete (they sit 88 lines away, outside the
chain, which is why the new code has to explain its own ordering), add the guard, and offer the
measurement with it rather than disputing the change.

Verification

check-natives.py 0 disagreements, 122 buildcheck tests green, and a search over TODO.md and
docs/benchmarks/ for bucket names, account ids, instance ids and presigned URLs finds nothing.

🤖 Generated with Claude Code

https://claude.ai/code/session_01E2h8gXyyE5UeimkL9vQv9G

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud