From 0d77592f50f0985bfb2b9eb7c9da35e751c381a8 Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Sat, 10 Oct 2026 00:01:31 +0200 Subject: [PATCH] handover: mark the B2 items already measured, and correct two assumptions about this machine The B2 block asked the local agent to discover seven things about the Windows CPU variants. Four of them were measured on that same workstation on 2026-10-09, so each item now says MEASURED or OPEN with the numbers, and the header says so too. Verifying a MEASURED line is still worth the minute it costs -- a different result there is a finding -- but the time belongs to the open ones. Measured, now stated: plain clang 23.1.3 builds all 14 variants with 0 errors; the directory holds 18 DLLs with nothing left in a Release/ subdirectory and jllama-files.txt naming 15 of them; verify-native-deps.py reports 18 libraries and 0 violations. Two assumptions in the draft do not hold on this machine, and both would have cost the local agent time: PATH carries clang 23.1.3, not 20.1.8, and Visual Studio ships no clang here at all (neither 2022\Community nor 18\BuildTools has VC\Tools\Llvm\x64) -- so the "newer versus older clang" choice the prompt set up does not exist; the 20.1.8 of the earlier comparison was unpacked separately. And jllama-extras.txt is ABSENT locally by design: CMake never writes it, merge-native-artifacts.sh derives it in CI from everything jllama-files.txt does not name. Asking "whether it exists" invites a false defect report, so the item now says what its absence means. Two things added that the measurements make useful rather than obvious. The Java suite is marked as the item that decides whether the variant path works at all, with the reason it is open: the measuring session's `mvn compile` dies in an Error Prone / NullAway crash on args/package-info.java, unrelated to the native work, and disabling annotation processing then breaks Lombok. And the variant-selection step carries a caveat before the agent spends time on it -- CLAUDE.md records that ggml logs module decisions at GGML_LOG_DEBUG only, so `-lv 4` may not name the chosen module; listing the DLLs the JVM has mapped is the fallback. The throughput item gets the static-build figures (pp512 391.6, tg128 82.3 t/s with clang and GGML_OPENMP=OFF) as the line a variant build should land near, so the number means something when it arrives. Committed on a branch off main rather than onto claude/hopeful-pascal-9jlbqb: that branch carries an unrelated open PR (#497, BoringSSL), and this file already lives on main, so it reaches the session branch on its next merge without mixing concerns into someone else's review. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01E2h8gXyyE5UeimkL9vQv9G --- docs/handover/local-agent-pr-a.md | 74 ++++++++++++++++++++++++------- 1 file changed, 59 insertions(+), 15 deletions(-) diff --git a/docs/handover/local-agent-pr-a.md b/docs/handover/local-agent-pr-a.md index 4a905cbd5..c2e54081b 100644 --- a/docs/handover/local-agent-pr-a.md +++ b/docs/handover/local-agent-pr-a.md @@ -5,6 +5,12 @@ AMD iGPU, MSVC + LLVM + Docker + JDK 21 + Maven). It describes what this branch what the remote session cannot verify from its sandbox, and the exact deliverables. The file is deleted again once PR A is merged. +**Deliverable 6 (B2) carries measurements taken on this same workstation on 2026-10-09**, marked +`MEASURED` or `OPEN` per item: four are done and only need confirming, three are genuinely open, and +the Java suite among them is the one that decides whether the variant path works at all. Verify a +`MEASURED` line rather than skipping it -- a different result there is a finding -- but spend the +time on the `OPEN` ones. + --- You are working in a checkout of `bernardladenthin/java-llama.cpp` on the branch @@ -62,21 +68,59 @@ Everything below is what only your machine can show. 6. **B2 pre-verification -- Windows CPU variants, before the CI job is written.** Build the variant library with plain clang (not `clang-cl`, not `cl.exe`: ggml drops 5 of the 14 variants under - `MSVC`, see CLAUDE.md "CPU variants"): - - Which clang: LLVM 20.1.8 from `PATH` versus the clang 22.x inside Visual Studio 2026. Try the - newer one first; patch `0017` is what makes `quants.c` compile there, so a failure on it is a - finding, not a toolchain problem. - - The exact `build.bat` command line that works (`-G "Ninja Multi-Config"`, `-DJLLAMA_CPU_VARIANTS=ON`, - the clang toolchain, `-DGGML_OPENMP=OFF`, `-DOS_NAME=Windows -DOS_ARCH=x86_64`), with and without - sccache on `PATH` (does the probe accept clang?). - - The resulting directory listing of `src/main/natives/net/ladenthin/llama/Windows/x86_64/cpu/` - (expect 18 DLLs: `jllama.dll`, `ggml.dll`, `ggml-base.dll`, 14 `ggml-cpu-.dll`, - `ggml-rpc.dll`) and the content of `jllama-files.txt`; whether `jllama-extras.txt` exists. - - `python .github/verify-native-deps.py` over that directory: paste its output. - - The Java suite (`mvn -f llama/pom.xml test`) against the variant build, with models. - - Which variant ggml picks on the 5800H (Zen 3 has AVX2 and no AVX-512, so expect `haswell`; the - loader log at `-lv 4` names it), and `llama-bench`-style numbers through the library if cheap: pp512 and - tg128 for Qwen3-0.6B Q4_K_M with 8 threads, variant build versus the single static CPU build. + `MSVC`, see CLAUDE.md "CPU variants"). + + **Four of these were already measured on this workstation on 2026-10-09, so VERIFY them rather + than discover them -- a different result is a finding worth reporting.** The three that are open + are marked as such, and the Java suite is the one that matters most, because it is the only step + that proves the variant path end to end and the only one the measuring session could not run (its + `mvn compile` dies in an Error Prone / NullAway crash on `args/package-info.java`, unrelated to + the native work; disabling annotation processing then breaks Lombok, so there was no quick + bypass). + + - **MEASURED -- which clang.** Plain clang **23.1.3** from `C:\Program Files\LLVM\bin` builds all + 14 variants with **0 errors**. Note two things the earlier draft of this prompt got wrong about + this machine: `PATH` carries **23.1.3**, not 20.1.8, and **Visual Studio ships no clang here** + (neither `2022\Community` nor `18\BuildTools` has a `VC\Tools\Llvm\x64`), so there is no "newer + versus older clang" choice to make -- the 20.1.8 used in the earlier comparison was unpacked + separately. 23.1.3 is the stricter of the two (it makes `-Wincompatible-pointer-types` an + error, which clang 22 introduced), and `patches/0017` is what makes `quants.c` compile there at + all: a failure on that file is a finding, not a toolchain problem. + - **OPEN -- the exact `build.bat` command line that works.** The measurement used `cmake` + directly, so `build.bat` itself is unverified for this configuration. Report the line that + works (`-G "Ninja Multi-Config"`, `-DJLLAMA_CPU_VARIANTS=ON`, the clang toolchain, + `-DGGML_OPENMP=OFF`, `-DOS_NAME=Windows -DOS_ARCH=x86_64`), with and without sccache on `PATH` + -- does `build.bat`'s probe accept clang, or does it fall back to an uncached build? The probe + only ever proved `cl.exe`, so this is genuinely unknown. + - **MEASURED -- the directory.** 18 DLLs in + `src/main/natives/net/ladenthin/llama/Windows/x86_64/cpu/`: `jllama.dll`, `ggml.dll`, + `ggml-base.dll`, 14 `ggml-cpu-.dll` (`x64`, `sse42`, `sandybridge`, `ivybridge`, + `piledriver`, `haswell`, `skylakex`, `cannonlake`, `cascadelake`, `icelake`, `cooperlake`, + `zen4`, `alderlake`, `sapphirerapids`) and `ggml-rpc.dll`, nothing in a `Release/` + subdirectory, and `jllama-files.txt` naming **15** of them -- the modules plus `ggml-rpc`. + **`jllama-extras.txt` is absent, and that is correct, not a defect:** CMake never writes it; + `merge-native-artifacts.sh` derives it in CI from everything in the directory that + `jllama-files.txt` does *not* name, which on Windows is exactly `ggml.dll` and `ggml-base.dll` + (they must be pre-loaded by full path -- there is no `$ORIGIN` on Windows). + - **MEASURED -- `python .github/verify-native-deps.py llama/src/main/natives`.** + "18 native libraries checked, 0 violations". Paste yours anyway; it is one line and it covers + the hybrid-CRT guard in both directions. + - **OPEN and the important one -- the Java suite** (`mvn -f llama/pom.xml test`) against the + variant build, with the models in place. CLAUDE.md's warning applies directly here: a + successful load proves little, because in the earlier Windows measurement *every* crash came + after a load that reported the right device count. So report the Surefire summary line, and + whether `RpcIntegrationTest` and the model-backed classes pass against a build whose CPU + backend is a loadable module rather than a linked library. + - **OPEN -- which variant ggml picks on the 5800H.** Zen 3 has AVX2 and no AVX-512, and + `alderlake` needs AVX-VNNI it does not have, so `haswell` is the expectation. Caveat before you + spend time on the log: CLAUDE.md records that ggml logs module decisions at `GGML_LOG_DEBUG` + only, so `-lv 4` may not name the chosen one. If it does not, list the DLLs the JVM actually + has mapped (Process Explorer, `listdlls`, or `tasklist /m`) instead of guessing. + - **OPEN -- pp512 and tg128** for Qwen3-0.6B Q4_K_M at 8 threads, variant build versus the single + static CPU build. For orientation, measured with `llama-bench` on this machine at + `GGML_NATIVE=ON` (so a single kernel, not the variant selection): clang static reached pp512 + 391.6 and tg128 82.3 t/s with `GGML_OPENMP=OFF`. A variant build landing near that means the + module indirection costs nothing; well below it is a finding. ## How to work on the branch