diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 39bf45b9..ab5b0cb7 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -2032,12 +2032,18 @@ jobs: with: distribution: 'temurin' java-version-file: .java-version - - name: Install Intel oneAPI (DPC++ + MKL) + - name: Install Intel oneAPI (DPC++ + MKL + oneDNN) + # oneDNN is required since llama.cpp #29107 (b11636): GGML_SYCL_DNN defaults to ON and + # ggml-sycl's CMakeLists now stops with FATAL_ERROR when DNNL is missing (it used to log + # "oneDNN not found" and build a module whose flash attention runs the slower MKL path). + # intel-oneapi-dnnl-devel is the oneAPI-bundled GPU build the error asks for -- not + # Ubuntu's CPU-only libdnnl-dev, which the same change rejects -- and setvars.sh puts it on + # CMAKE_PREFIX_PATH. The Windows SYCL job installs the matching intel.oneapi.win.dnnl. run: | wget -qO- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB | gpg --dearmor | sudo tee /usr/share/keyrings/oneapi-archive-keyring.gpg > /dev/null echo "deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main" | sudo tee /etc/apt/sources.list.d/oneAPI.list sudo apt-get update - sudo apt-get install -y intel-oneapi-compiler-dpcpp-cpp intel-oneapi-mkl-devel + sudo apt-get install -y intel-oneapi-compiler-dpcpp-cpp intel-oneapi-mkl-devel intel-oneapi-dnnl-devel - name: Build libraries shell: bash run: | diff --git a/CLAUDE.md b/CLAUDE.md index 8125c3ed..b0cf2939 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b11632** +Current llama.cpp pinned version: **b11639** ## Natives jars: library jars and GPU module jars (`.github/natives.csv`) @@ -178,7 +178,7 @@ measurements behind the Windows step are in `docs/handover/local-agent-report-b1 `add_subdirectory(ggml)` when a `ggml` target exists; a module-only build never adds llama.cpp). llama, mtmd and llama-common stay static in `libjllama.so`. **Every ggml default llama.cpp's own CMakeLists changes before its `add_subdirectory(ggml)` is repeated** -- `GGML_LLAMAFILE_DEFAULT` - and `GGML_CUDA_GRAPHS_DEFAULT` at b11632 -- because adding ggml first skips that block; re-check with + and `GGML_CUDA_GRAPHS_DEFAULT` at b11639 -- because adding ggml first skips that block; re-check with `grep -n _DEFAULT CMakeLists.txt` at the new tag on every bump. The second one was missing once: ggml's own default for CUDA graphs is OFF, and a CUDA module built without it lost two thirds of its token generation (265.6 -> 88.8 tg128 on an RTX 3070, prompt processing -20%; Vulkan, which @@ -930,7 +930,11 @@ jar barely shrinks (zip already compressed the code); what shrinks is the extrac **SYCL is fp16 on both OSes, one jar each**, as upstream's SYCL release builds (`GGML_SYCL_F16=ON`). The second Linux precision (`sycl-fp32-linux-x86-64`, fp32 accumulation) was dropped with the modular natives: nobody had asked for it, and two modules of the same backend on one classpath would both -register the same device. +register the same device. **Both SYCL jobs install oneDNN** (Linux `intel-oneapi-dnnl-devel`, Windows +`intel.oneapi.win.dnnl`): since llama.cpp #29107 (b11636) `GGML_SYCL_DNN` (default ON) fails the configure +when oneDNN is missing or is a CPU-only build, where it used to warn and build a module whose flash attention +runs the slower MKL path -- which is what the Linux module had been until the b11639 bump. The module then +needs `libdnnl.so.3` / `dnnl.dll` at run time, from the same oneAPI environment (`setvars.sh`) as MKL. The Linux jobs that install a multi-GB vendor toolchain (CUDA, ROCm, SYCL) start with `ggml-org/free-disk-space` — the same guard upstream llama.cpp's CUDA/ROCm jobs use (ROCm also clears the @@ -1107,7 +1111,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network for the asset build; the embed step is plain cmake -P -git clone --depth 1 --branch b11632 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b11639 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build ) mkdir -p webui-generated /tmp/ui-gen cmake -DUI_SOURCE_DIR=/tmp/lc/tools/ui -DUI_BINARY_DIR=/tmp/ui-gen \ @@ -1147,7 +1151,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`JLLAMA_GIT_TAG b11632`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`JLLAMA_GIT_TAG b11639`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -1381,8 +1385,8 @@ Current patches: | `0015-rpc-embeddable-client-and-stoppable-server.patch` | **Makes ggml-rpc usable inside a JVM** (see "RPC backend" below). Upstream treats every client-side RPC problem as `GGML_ABORT` — a malformed endpoint, a server that is not running, a failed handshake — which in a JVM kills the application over a typo in `--rpc`. **(1)** The *registration* path reports failure instead: `rpc_dispatcher::start()` returns `false`, `try_get_dispatcher()` `nullptr`, `ggml_backend_rpc_get_device_count()` `0`, `ggml_backend_rpc_add_server()` `nullptr`, and `add_rpc_devices()` (`common/arg.cpp`) throws `std::invalid_argument` naming the server where it used to register `nullptr` and silently run without it. Every path *after* registration keeps the original contract through `get_dispatcher()`, which still aborts — a server that vanishes mid-inference stays fatal (TODO). **(2)** `add_server()`'s cache hit is re-checked (a stopped server used to pass registration and abort on the first tensor upload), and `ggml_backend_rpc_get_device_memory()` of a gone server returns 0/0 instead of aborting — RPC devices stay in ggml's process-wide registry forever, and `common_init()` queries the memory of **every** registered device at `-lv 4`, so a stopped server would otherwise take down an unrelated later load. **(3)** The server becomes stoppable: `ggml_backend_rpc_stop_server()` (disconnects the client being served, wakes `accept()` with a loopback connection — the one portable way) and `ggml_backend_rpc_server_listening()`; the loop now reaches its cleanup and frees its backends on every early return, but deliberately **not** `rpc_transport_shutdown()`, whose `WSACleanup()` would invalidate every other RPC socket in the process. **(4)** transport: `MSG_NOSIGNAL`/`SO_NOSIGPIPE`, `socket_t::shutdown()`, and the fds `connect()`/`create_server()`/`accept()` leaked on their error paths are closed. Touches `ggml/include/ggml-rpc.h`, `ggml/src/ggml-rpc/{ggml-rpc.cpp,transport.cpp,transport.h}` and one function in `common/arg.cpp` (`add_rpc_devices`, away from `0001`'s `common_params_parse` hunks). **Runnable guard: `src/test/cpp/test_rpc.cpp`** — links both new functions (a bump that drops the patch fails `C++ Tests` at link time on every platform) and drives registration, stop and the memory query over loopback. Upstream-submittable; **not yet filed upstream**. On a bump, check whether upstream grew its own stop function or non-aborting registration (grep `stop_server` / `GGML_ABORT("Failed to connect` in `ggml-rpc.cpp`). **Refreshed at the b11450 bump** (#26610, `-sm tensor` over RPC, ~800 lines in `ggml-rpc.cpp`, protocol 7 → 8): three hunks lost their context -- new dispatcher members beside `start()`, and new `get_proc_address` names before `return NULL` (the two new names now sit after `ggml_backend_comm_allreduce_tensor`); the `+`/`-` lines are unchanged. The server-to-server comm #26610 adds (a listener on `0.0.0.0`, an `accept()` a stop does not wake) is on file in `TODO.md`. **Refreshed at the b11558 bump** (#29544 turns `GGML_RPC_DEBUG` into a verbosity level 0-3: one `ggml-rpc/log.h` with `LOG_ERROR`/`LOG_WARN`/`LOG_INFO` and `LOG_DBG`/`LOG_DBG2`/`LOG_DBG3`, every `GGML_LOG_*`, `printf` and `fprintf(stderr)` of the backend routed through it, and `transport_name()` where `shutdown()` goes in `transport.h`): 12 of the 24 hunks lost their context and were regenerated against the new text; the changes are the same, and the patch's own five error lines (the HELLO handshake in `negotiate_hello`, four in `rpc_dispatcher::start()`) use `LOG_ERROR` now like the code around them (`LOG_ERROR` is `GGML_LOG_ERROR`). One consequence outside the patch: the server's `Accepted client connection` line is `LOG_INFO` through ggml's log callback now (stderr by default, a `setLogger` sink at verbosity 4), no longer `printf` to stdout; `smoke-rpc.sh` captures both streams into `rpc-server.log`, so its grep is unaffected. | | `0017-ggml-cpu-x86-prefetch-byte-offset-and-msvc-type.patch` | **Makes `GGML_CPU_ALL_VARIANTS=ON` buildable with plain clang on Windows, and fixes a prefetch address that is wrong everywhere.** The four `_mm_prefetch` calls in `ggml/src/ggml-cpu/arch/x86/quants.c` (the SSSE3-without-AVX branch of `ggml_vec_dot_q4_0_q8_0`, and the only `_mm_prefetch` calls in the whole ggml tree) carry two independent defects; both are fixed by routing the address through one new `static inline` helper in the new header `ggml/src/ggml-cpu/arch/x86/prefetch.h`, which is included relatively, so **no CMake change is needed**. **(1) Type -- fatal only on a current clang:** MSVC declares the intrinsic as `void _mm_prefetch(char const *, int)`; clang's own `xmmintrin.h` defines a casting macro but only `#ifndef _MSC_VER`, so on a `*-windows-msvc` target clang leaves the declaration to Microsoft and a typed pointer is `-Wincompatible-pointer-types`. **clang 22 promoted that to an error by default** ([llvm-project #157364](https://github.com/llvm/llvm-project/pull/157364); the [22.1.0 release notes](https://releases.llvm.org/22.1.0/tools/clang/docs/ReleaseNotes.html) add that `-Wno-error=incompatible-pointer-types` downgrades it) -- *not* clang 16, which promoted only `-Wincompatible-FUNCTION-pointer-types`; an earlier version of this row said otherwise. Measured on the real file at `-msse4.2`, unpatched → patched: clang **20.1.8** 4 warnings → 0/0, clang **22.1.3** 4 errors → 0/0, clang **23.1.3** 4 errors → 0/0. Plain clang is exactly what `cmake/x64-windows-llvm.cmake` selects -- and plain clang is the only way to get all **14** CPU variants, because `clang-cl` makes CMake's `MSVC` true and the `if (NOT MSVC)` block in `ggml/src/CMakeLists.txt` then drops `zen4`, `cooperlake`, `sapphirerapids`, `ivybridge` and `piledriver`. **(2) Arithmetic:** `&x[ib] + sizeof(block_q4_0)` adds `sizeof()` *elements*, not bytes -- +324 and +1156 instead of +18 and +34 (the sizes `ggml-common.h` pins by `static_assert`), and +648 instead of +36 for the `2 *` pair. That half is wrong on every platform, including where the call compiles. **Measured:** no throughput difference (see the test-table row), so this is a correctness and portability fix, not a performance one. **Upstream-submittable and meant to be submitted.** **Why upstream's own build is green is measured, not an open question:** their `windows-cpu` release job builds exactly this with plain clang, `GGML_CPU_ALL_VARIANTS=ON` and no suppressing flag (`CMAKE_ARGS` carries only build-target toggles), and ships `ggml-cpu-sse42.dll` -- because the `windows-2025-vs2026` image has **LLVM 20.1.8** on `PATH`, where these are warnings. That is narrower than it looks: the same image also ships **clang 22.1.3** inside Visual Studio 2026 (`VC/Tools/Llvm/x64`), which *fails* on the unpatched file -- upstream reaches the working compiler only because `C:\Program Files/LLVM/bin` comes first on `PATH`. It is **not** a header difference: all three MSVC toolsets on that image (14.29.30133, 14.44.35207, 14.51.36231) declare it identically as `extern void _mm_prefetch(char const*_A, int _Sel);` -- that was the first hypothesis, and a one-off probe workflow (deleted again once it had answered) disproved it by printing the declaration from each toolset on the image. So this patch is **ahead of** upstream rather than diverging: their build breaks the moment the image ships a clang that makes the diagnostic fatal, and until then they compile four warnings and a prefetch address 18x/34x too far. **Runnable guard: `src/test/cpp/test_prefetch.cpp`.** On a bump, run `git grep -n _mm_prefetch ggml/src/ggml-cpu/arch/x86/quants.c` at the new tag: if the call sites already cast **before** the addition, **drop this patch rather than refreshing it**. | | `0016-model-kolibri1.patch` | **Adds Aleph Alpha's Kolibri-1 (architecture `kolibri1`) ahead of upstream** -- a 78B German/English reasoning MoE (3.46B active) that upstream does not support yet ([ggml-org/llama.cpp#29922](https://github.com/ggml-org/llama.cpp/issues/29922), no PR at the time). A **temporary carry, not a fix**: drop it the moment upstream registers the architecture (`git grep -n kolibri src/llama-arch.cpp` at the new tag), and keep `test_kolibri1.cpp` -- its numerical comparisons must then pass against upstream's implementation, while its GGUF-format rows follow whatever format upstream's converter fixes (see the test-table row). **Architecture** (Aleph Alpha's vLLM reference, `aleph_alpha_inference/kolibri1.py` at `049a6a7`): GQA with per-head q/k RMSNorm; `layer_types` interleaves sliding-window layers (NEOX RoPE) with full-attention layers that have **no** positional encoding; sandwich norms around attention and MoE; every layer routed MoE + one ungated shared expert; the router selects top-k on **`logits + expert_bias`** and weights by the **unbiased `sigmoid(logits)`** -- not DeepSeek-V3's sigmoid router (selection on `sigmoid(logits) + bias`), which picks other experts as soon as the bias is non-zero. The graph therefore builds the selection itself and passes it to `build_moe_ffn` (`selected_experts_in`), leaving upstream's shared MoE code untouched. **Derived from two community ports, which are incompatible with each other's GGUFs** -- Eliasfpv28's AFMoE-based one (base b11378, `expert_gating_func` 2, pre-tokenizer `qwen2`) and the Qwen3-MoE-based one behind Hob-forge's GGUFs (base b11381, as carried in [mjwsolo/localcode#101](https://github.com/mjwsolo/localcode/pull/101): gating 5, pre-tokenizer `kolibri1`, rejects gating 2). This patch **loads both dialects**: gating 2, 5 or absent (all mean the one router), `kolibri1` mapped to the Qwen2 pre-tokenizer in `llama-vocab.cpp`, `expert_weights_norm` / `expert_shared_count` / `output.weight` optional. Every source, with license, is listed in the patch header; `REUSE.toml` annotates the file `MIT AND Apache-2.0` (Eliasfpv28's additions are Apache-2.0). Touches `src/llama-arch.{h,cpp}`, `src/llama-model.cpp` (mapping + NEOX rope list), `src/llama-vocab.cpp` (one name), `src/models/models.h` and the new `src/models/kolibri1.cpp`; no converter. **Runnable guard: `src/test/cpp/test_kolibri1.cpp`** (writes tiny random GGUFs in both dialects and compares every logit with an independent double-precision reference). **Verified against the real 78B model on 2026-10-11** (rented 16-vCPU / 123 GB instance, both published dialects fetched on the machine): both load, and a chat turn through the model's own template answers correctly in each -- `content` `"Berlin."` for a German question and `"51"` for 17x3, with the chain of thought in `reasoning_content` and `finish_reason` `stop`. **The two dialects tokenize identically** (`[452 22090 493 1678 2459 5513 46]`, round-tripping to the input sentence), which is the direct evidence that mapping `kolibri1` to the Qwen2 pre-tokenizer is right. Q4_K_M (44 GB) gave 13.4 t/s and Q3_K_S (31 GB) 14.6 t/s of generation on 16 cores. **Two traps for whoever repeats this:** a *raw* completion on this instruct model degenerates into repetition whatever the patch does, so measure through the chat endpoint; and `chatCompleteText` returns only `content`, which is **empty** when a reasoning model spends its token budget inside the thinking block -- use `handleChatCompletions` and a budget in the hundreds. **Still not verified:** GPU backends -- the community ports report real-model checks (top-1 67/67 against a float64 reference for Q8_0, tool calls, German text). | -| `0018-ggml-amx-free-the-aligned-buffer.patch` | **The AMX weight buffer is freed with `free()` but allocated with `_aligned_malloc()` on Windows -- `STATUS_HEAP_CORRUPTION` (0xC0000374) the moment a model is unloaded.** `ggml_backend_amx_buffer_type_alloc_buffer()` (`ggml/src/ggml-cpu/amx/amx.cpp`) allocates with `ggml_aligned_malloc()`, which is `_aligned_malloc()` under `_MSC_VER` (plain clang on a `*-windows-msvc` target defines it too); `ggml_backend_amx_buffer_free_buffer()` called `free()` on it. Fine on Linux (`posix_memalign` + `free`), fatal on Windows, and only there. The patch frees with `ggml_aligned_free(buffer->context, buffer->size)`, exactly what `ggml_backend_cpu_buffer_free_buffer()` does; on Linux that is `free()`, so nothing changes there. **Reached by** the `sapphirerapids` CPU module (the only variant built with `AMX_TILE AMX_INT8`, which `ggml_backend_load_best` picks on Sapphire/Emerald/Granite Rapids; `ggml_amx_init()` is `return true` on `_WIN32`) and a model whose weights have AMX kernels (Q4_0, Q4_1, Q8_0, Q4_K, Q5_K, Q6_K, IQ4_XS -- i.e. every CI model). **Found by the second full run of the modular natives** (38069595101): `Java Tests Windows 2025 x86_64 (clang)` landed on an `Intel(R) Xeon(R) 6973P-C` runner (Granite Rapids), loaded `ggml-cpu-sapphirerapids.dll`, ran `GrammarExample`'s two generations fine and died with exit code -1073740940 at `LlamaModel.close()`; the first run (38065017595) had the same job green on an AMD EPYC 7763 (no AVX-512, `haswell`). **The Windows runners are a CPU lottery**, so which CPU module a Windows job exercises differs per run; before the modular natives the Windows jar was a static Haswell-level build and this code never ran. **Of the six callers of `ggml_aligned_malloc` in the tree this was the only mismatched pair** (`ggml-backend.cpp`, `ggml-cpu.c` twice, `ggml-openvino.cpp`, `ggml-zdnn.cpp` and `ggml.c` all pair correctly), and `ggml-backend.cpp:2464` is the line the fix copies. **Reproduced deliberately rather than waiting for the lottery again**: on a rented Granite Rapids machine, one CPU module staged at a time with the same model and the DLLs CI itself built, `sapphirerapids` crashed **6 of 6** runs -- including at **one** thread, which rules out a data race -- while `cooperlake`, `icelake` and `haswell` were clean 6 of 6, and on Linux with the same CPU all six exit 0 with byte-identical output. The crash lands *after* the inference prints its timings, in cleanup: the signature of a heap corrupted earlier and detected at `free`. With the patch applied the same matrix on the same machine is **clean in all twelve runs**. **The guard is a configure-time check, not a `jllama_test` case**: the AMX code exists only in that one variant of a `GGML_CPU_ALL_VARIANTS` build and executes only on AMX hardware, which the static test build, the local agent's Zen 3 and the cloud sandboxes lack -- so **`llama/cmake/check-aligned-free-pairing.cmake`** fails the configure on **every** platform when a ggml source calls `ggml_aligned_malloc` and never `ggml_aligned_free` (verified red against the unpatched source, naming `amx.cpp`, and green with this patch). A run-time test cannot replace it: without an AMX CPU there is nothing to observe, which is exactly how this shipped. What exercises it: the Windows x86-64 Java test job and the `windows-x86-64` smoke row whenever GitHub hands them an Intel AMX runner; the patch itself is held by `check-patches.py`, applies at b11632, and the patched file compiles with the variant's flags (GCC 13, `-mamx-tile -mamx-int8 -mavx512vnni ...`). Upstream master still carried the `free()` on 2026-10-10; **not yet filed upstream** (nothing before the release). **Drop-check on every bump:** `grep -n "free(buffer->context)" ggml/src/ggml-cpu/amx/amx.cpp` at the new tag -- gone means upstream fixed it, drop the patch. | -| `0019-ggml-cpu-select-the-neon-fp32-to-fp16-conversion.patch` | **Every ARM build converts fp32 to fp16 in software, ours and upstream's.** The `__ARM_NEON` branch of `simd-mappings.h` defines both `GGML_CPU_COMPUTE_*` macros and selects only `FP16_TO_FP32`, so `GGML_CPU_FP32_TO_FP16` falls through to `GGML_COMPUTE_FP32_TO_FP16` -- ggml-base's deliberately portable software bit manipulation -- while `neon_compute_fp32_to_fp16`, one `fcvt`, stays defined and referenced by nothing. The `GGML_CPU_*` pair exists to be the *accelerated* variant; on ARM it was defined to be the unaccelerated one. `__POWER9_VECTOR__` and `__riscv_zfhmin` select both, and the comment on that fallback has always promised NEON does too. **Not a regression** -- absent in all 25 most recent commits touching the file, back to 2025-04-10. **Why only ARM feels it:** `ggml_cpu_fp32_to_fp16`, the `from_float` of `GGML_TYPE_F16`, has vector paths for F16C, RISC-V and s390x (the last from #30157) and **none** for NEON, so there the whole function is the scalar loop through the macro. **In the shipped artifacts:** no `fcvt` of any form in that function in any of this project's 17 ARM libraries (`cpu-linux-aarch64` 8 variants, `cpu-android-aarch64` 7, `cpu-windows-aarch64`, `metal-macos-aarch64`) nor in upstream's own `bin-ubuntu-arm64` / `bin-macos-arm64`; a plain `gcc` with no flags defines `__ARM_NEON` and `__ARM_FP16_FORMAT_IEEE` on aarch64, so nothing can miss it. **Measured 2.45x-13.3x** on five ARM core types, three operating systems and nine compiler builds, with the opposite direction in the same run as a control (within 3.2 % everywhere); **bit-exact over all 2^32 float inputs**, differing only in NaN payload, where x86 F16C gives the same numbers -- so it makes ARM agree with what x86 already ships. **Not an inference speed-up:** `perf` puts 83.9 % of an F16 `pp512` run in tinyBLAS's gemm, which converts in its own microkernel, and 0.03 % here. The patch also replaces the stale comment above the fallback with an `#error` pair that refuses a branch defining a COMPUTE macro without selecting it -- its position chosen by a mutation measurement, not by taste (4 of 4 correctness cases caught before the fallback against 3 of 4 directly after the chain). **The `FP16_TO_FP32` half carries `&& !defined(__F16C__)`, and that carve-out is load-bearing from the next bump on:** upstream #30343 (`685f70894`, merged 2026-10-14, *after* the b11632 pin) puts that one direction back on the lookup table deliberately -- clang lowers the F16C intrinsic into the live accumulator register of the quantized dot products and serialises the loop, 45-65 % slower Q8_0/Q4_0 `MUL_MAT_ID` -- so without it the guard fires on **11 of the 14 x86-64 CPU variant modules**. At the pin #30343 is absent, the `__F16C__` block there still selects both directions, and the carve-out is inert, so x86 behaviour at the pin is exactly upstream's. A **first** version of this patch also moved #30270's two F16C selections into the `__F16C__` branch; that was dropped when #30343 turned the block into a documented exception with an issue link, which is also why this diff now applies **both** at the pin and on upstream master (verified with `git apply --check` against each, and `clang -E` silent in all five configurations of the patched pinned source). Full record: [`docs/benchmarks/arm-fp16-conversion.md`](docs/benchmarks/arm-fp16-conversion.md). **Submitted upstream as [ggml-org/llama.cpp#30505](https://github.com/ggml-org/llama.cpp/pull/30505)** (open, rebased onto master 2026-10-14 at the maintainer's request), byte-identical to this diff -- the diff body of the patch file and `git diff` of the PR commit hash the same. **Drop-check on every bump:** `git grep -n "GGML_CPU_FP32_TO_FP16(x) GGML_CPU_COMPUTE_FP32_TO_FP16" ggml/src/ggml-cpu/simd-mappings.h` at the new tag -- if the `__ARM_NEON` branch carries that line, **drop this patch rather than refreshing it**. | +| `0018-ggml-amx-free-the-aligned-buffer.patch` | **The AMX weight buffer is freed with `free()` but allocated with `_aligned_malloc()` on Windows -- `STATUS_HEAP_CORRUPTION` (0xC0000374) the moment a model is unloaded.** `ggml_backend_amx_buffer_type_alloc_buffer()` (`ggml/src/ggml-cpu/amx/amx.cpp`) allocates with `ggml_aligned_malloc()`, which is `_aligned_malloc()` under `_MSC_VER` (plain clang on a `*-windows-msvc` target defines it too); `ggml_backend_amx_buffer_free_buffer()` called `free()` on it. Fine on Linux (`posix_memalign` + `free`), fatal on Windows, and only there. The patch frees with `ggml_aligned_free(buffer->context, buffer->size)`, exactly what `ggml_backend_cpu_buffer_free_buffer()` does; on Linux that is `free()`, so nothing changes there. **Reached by** the `sapphirerapids` CPU module (the only variant built with `AMX_TILE AMX_INT8`, which `ggml_backend_load_best` picks on Sapphire/Emerald/Granite Rapids; `ggml_amx_init()` is `return true` on `_WIN32`) and a model whose weights have AMX kernels (Q4_0, Q4_1, Q8_0, Q4_K, Q5_K, Q6_K, IQ4_XS -- i.e. every CI model). **Found by the second full run of the modular natives** (38069595101): `Java Tests Windows 2025 x86_64 (clang)` landed on an `Intel(R) Xeon(R) 6973P-C` runner (Granite Rapids), loaded `ggml-cpu-sapphirerapids.dll`, ran `GrammarExample`'s two generations fine and died with exit code -1073740940 at `LlamaModel.close()`; the first run (38065017595) had the same job green on an AMD EPYC 7763 (no AVX-512, `haswell`). **The Windows runners are a CPU lottery**, so which CPU module a Windows job exercises differs per run; before the modular natives the Windows jar was a static Haswell-level build and this code never ran. **Of the six callers of `ggml_aligned_malloc` in the tree this was the only mismatched pair** (`ggml-backend.cpp`, `ggml-cpu.c` twice, `ggml-openvino.cpp`, `ggml-zdnn.cpp` and `ggml.c` all pair correctly), and `ggml-backend.cpp:2464` is the line the fix copies. **Reproduced deliberately rather than waiting for the lottery again**: on a rented Granite Rapids machine, one CPU module staged at a time with the same model and the DLLs CI itself built, `sapphirerapids` crashed **6 of 6** runs -- including at **one** thread, which rules out a data race -- while `cooperlake`, `icelake` and `haswell` were clean 6 of 6, and on Linux with the same CPU all six exit 0 with byte-identical output. The crash lands *after* the inference prints its timings, in cleanup: the signature of a heap corrupted earlier and detected at `free`. With the patch applied the same matrix on the same machine is **clean in all twelve runs**. **The guard is a configure-time check, not a `jllama_test` case**: the AMX code exists only in that one variant of a `GGML_CPU_ALL_VARIANTS` build and executes only on AMX hardware, which the static test build, the local agent's Zen 3 and the cloud sandboxes lack -- so **`llama/cmake/check-aligned-free-pairing.cmake`** fails the configure on **every** platform when a ggml source calls `ggml_aligned_malloc` and never `ggml_aligned_free` (verified red against the unpatched source, naming `amx.cpp`, and green with this patch). A run-time test cannot replace it: without an AMX CPU there is nothing to observe, which is exactly how this shipped. What exercises it: the Windows x86-64 Java test job and the `windows-x86-64` smoke row whenever GitHub hands them an Intel AMX runner; the patch itself is held by `check-patches.py`, applies at b11639, and the patched file compiles with the variant's flags (GCC 13, `-mamx-tile -mamx-int8 -mavx512vnni ...`). Upstream master still carried the `free()` on 2026-10-10; **not yet filed upstream** (nothing before the release). **Drop-check on every bump:** `grep -n "free(buffer->context)" ggml/src/ggml-cpu/amx/amx.cpp` at the new tag -- gone means upstream fixed it, drop the patch. | +| `0019-ggml-cpu-select-the-neon-fp32-to-fp16-conversion.patch` | **Every ARM build converts fp32 to fp16 in software, ours and upstream's.** The `__ARM_NEON` branch of `simd-mappings.h` defines both `GGML_CPU_COMPUTE_*` macros and selects only `FP16_TO_FP32`, so `GGML_CPU_FP32_TO_FP16` falls through to `GGML_COMPUTE_FP32_TO_FP16` -- ggml-base's deliberately portable software bit manipulation -- while `neon_compute_fp32_to_fp16`, one `fcvt`, stays defined and referenced by nothing. The `GGML_CPU_*` pair exists to be the *accelerated* variant; on ARM it was defined to be the unaccelerated one. `__POWER9_VECTOR__` and `__riscv_zfhmin` select both, and the comment on that fallback has always promised NEON does too. **Not a regression** -- absent in all 25 most recent commits touching the file, back to 2025-04-10. **Why only ARM feels it:** `ggml_cpu_fp32_to_fp16`, the `from_float` of `GGML_TYPE_F16`, has vector paths for F16C, RISC-V and s390x (the last from #30157) and **none** for NEON, so there the whole function is the scalar loop through the macro. **In the shipped artifacts:** no `fcvt` of any form in that function in any of this project's 17 ARM libraries (`cpu-linux-aarch64` 8 variants, `cpu-android-aarch64` 7, `cpu-windows-aarch64`, `metal-macos-aarch64`) nor in upstream's own `bin-ubuntu-arm64` / `bin-macos-arm64`; a plain `gcc` with no flags defines `__ARM_NEON` and `__ARM_FP16_FORMAT_IEEE` on aarch64, so nothing can miss it. **Measured 2.45x-13.3x** on five ARM core types, three operating systems and nine compiler builds, with the opposite direction in the same run as a control (within 3.2 % everywhere); **bit-exact over all 2^32 float inputs**, differing only in NaN payload, where x86 F16C gives the same numbers -- so it makes ARM agree with what x86 already ships. **Not an inference speed-up:** `perf` puts 83.9 % of an F16 `pp512` run in tinyBLAS's gemm, which converts in its own microkernel, and 0.03 % here. The patch also replaces the stale comment above the fallback with an `#error` pair that refuses a branch defining a COMPUTE macro without selecting it -- its position chosen by a mutation measurement, not by taste (4 of 4 correctness cases caught before the fallback against 3 of 4 directly after the chain). **The `FP16_TO_FP32` half carries `&& !defined(__F16C__)`, and that carve-out is load-bearing since the b11639 bump:** upstream #30343 (`685f70894` = b11635, merged 2026-10-14) puts that one direction back on the lookup table deliberately -- clang lowers the F16C intrinsic into the live accumulator register of the quantized dot products and serialises the loop, 45-65 % slower Q8_0/Q4_0 `MUL_MAT_ID` -- so without it the guard fires on **11 of the 14 x86-64 CPU variant modules**. Up to b11632 #30343 was absent, the `__F16C__` block still selected both directions and the carve-out was inert; from b11639 on the pin carries #30343, so x86 FP16->FP32 is on the lookup table exactly as upstream ships it, and the carve-out is what keeps the guard quiet there (`gcc -E` of the patched b11639 header silent for `-march=x86-64`, `-msse4.2`, `-march=haswell`, `-march=sapphirerapids`, `-march=znver4` and `-mavx -mf16c`). A **first** version of this patch also moved #30270's two F16C selections into the `__F16C__` branch; that was dropped when #30343 turned the block into a documented exception with an issue link, which is also why this diff now applies **both** at the pin and on upstream master (verified with `git apply --check` against each, and `clang -E` silent in all five configurations of the patched pinned source). Full record: [`docs/benchmarks/arm-fp16-conversion.md`](docs/benchmarks/arm-fp16-conversion.md). **Submitted upstream as [ggml-org/llama.cpp#30505](https://github.com/ggml-org/llama.cpp/pull/30505)** (open, rebased onto master 2026-10-14 at the maintainer's request), byte-identical to this diff -- the diff body of the patch file and `git diff` of the PR commit hash the same. **Drop-check on every bump:** `git grep -n "GGML_CPU_FP32_TO_FP16(x) GGML_CPU_COMPUTE_FP32_TO_FP16" ggml/src/ggml-cpu/simd-mappings.h` at the new tag -- if the `__ARM_NEON` branch carries that line, **drop this patch rather than refreshing it**. | **Dropped patches** (`0004`, `0005`, `0009`, `0010`, `0011`, `0012`, `0013` -- upstream fixed each defect itself) are recorded in [`docs/history/dropped-llama-patches.md`](docs/history/dropped-llama-patches.md): @@ -2420,7 +2424,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `JLLAMA_GIT_TAG b11632`. +llama.cpp is fetched via CMake FetchContent, pinned to `JLLAMA_GIT_TAG b11639`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.18.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index b9a912d5..0b760a57 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b11632](https://img.shields.io/badge/llama.cpp-%23b11632-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11632) +[![llama.cpp b11639](https://img.shields.io/badge/llama.cpp-%23b11639-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11639) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index b924be4b..10bcb440 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -892,3 +892,5 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b11612–b11617 | patches + upstream verification | **All thirteen patches apply unchanged at b11617** (`git apply` in order on a pristine b11617 worktree, 57 modified files; `check-patches.py`: 13 patches, 101 hunks, 0 problems; the real applier on a fresh `FetchContent` checkout -- `rm -rf build` -- with stamp head `14310190c`), then the static test build, `ctest` **608/608**, and the model-free native Java tests over the library (`NativeLibraryLoadSmokeTest` with the `LlamaCppVersion` pin against the linked `b11617-`, `LlamaLoggerTest`, `RpcServerTest`, `BackendLoadTest`: 27/27). Drop-checks against pristine b11617 are those of the b11612 row, the range touching no patch-target file: `split_sum` unguarded at `src/llama-model.cpp:1648` (`0012`; the upstream fix `df18bee11` / #29677 is still in no tag -- newest b11618 -- so the drop rule in `TODO.md` keys on the fixing commit now, not on "past b11612"), no `kolibri` (`0016`), no `common_log` hook (`0014`), no `stop_server` (`0015`), uncast `_mm_prefetch` (`0017`), `free(buffer->context)` (`0018`), the software fp16 conversion on `__ARM_NEON` (`0019`), the `GetCommandLineW` override and no `common_params_parse_main` (`0001`), the unconditional load-progress assignment (`0002`), no `LLAMA_SERVER_WORKER_CMD` (`0008`), no `get_slot_prompt_similarity` (`0003`). `check-natives.py`: 23 natives jars, 0 disagreements. Not run locally: the model-backed Java suite and the modular build -- CI covers both. | | b11617–b11632 | 15 commits (9 to `df18bee11` = b11626, 6 more to master's head of 2026-10-14, `62df1815b` = b11632 -- pinned through `JLLAMA_GIT_COMMIT` ahead of the tag), 43 files, ~1,250/180 lines (no `tools/ui`, no `.github/workflows/release.yml`, root `CMakeLists.txt` unchanged). **`src/llama-model.cpp` (#29677, `df18bee11`): the one-line even-split fallback in `load_tensors()` -- the defect `patches/0012` carried, fixed upstream; see the patches row.** b11626–b11632 on top: `common/common.h` + `common/arg.cpp` (#30459 `--spec-draft-n-ubatch`, a new `speculative.draft.n_ubatch` field and server option -- additive, not exposed in `ModelOption` yet, and away from `0001`'s hunks, which apply unchanged), `ggml/include/ggml.h` (#30501, a comment on `mul_mat_id` duplicate ids), `ggml/src/ggml.c` + `tests/test-conv-1d.cpp` (#28738 conv_1d result layout for batched input -- a graph-level fix in the CPU path, no API change), `ggml-cuda` (#29353 GDN chunked kernel, `gated-delta-net-mma.cu` new: the CUDA module jar), a code-review skill and docs. `vendor/cpp-httplib/` (#30494, cpp-httplib 0.61.0: `httplib.{h,cpp}`, +111/-34; `BORINGSSL_VERSION` still `0.20260929.0`) -- `jllama` links the `cpp-httplib` target, the server and the download code recompile against it, nothing here names its internals. `ggml/src/ggml-rpc/` (#30481 "improve RPC tensor transfer logs": a `rpc_log` helper with its own `log.cpp`, the transfer sizes at `LOG_DBG2`, two `LOG_WARN`) -- `0015` applies unchanged, its hunks sit outside the logging lines. `common/jinja/value.cpp` (#30362, floats in `tojson` written as Python does), `ggml-cuda` (#30325 SIMT conv2d on AMD, #28195 RDNA3.5 MMQ tiles), `ggml-opencl` (#30365, #30418: Adreno MoE), six `src/models/*` embedding graphs (#30461 drops the `t_embd` expand). Nothing on the priority list changed (`common/common.h`, `chat.h`, `arg.h`, `llama.h`, `log.h`, `tools/server/*.h`, `mtmd*.h`, `ggml/include` all untouched); toolchain pins unchanged. **No project source change** beyond the guard that leaves with `0012` (`test_model_split.cpp`, its CMake line and the `${llama.cpp_SOURCE_DIR}/src` include dir only it needed). | | b11617–b11632 | patches + upstream verification | **`0012` dropped, twelve patches apply unchanged** (`git apply` in order on pristine `df18bee11` and `62df1815b` worktrees, 54 modified files; `check-patches.py`: 12 patches, 95 hunks, 0 problems). **The drop-check fired both ways:** `src/llama-model.cpp:1650` is the guarded `splits[i] = split_sum != 0.0f ? ... : float(i + 1) / n_devices()` now, and `git apply --check` of `0012` on the `df18bee11` tree fails at `src/llama-model.cpp:1638` -- upstream's line is inside the block the patch replaced, so unlike `0010` this one could not have been carried on by mistake. **The first bump pinned ahead of the tag**: no tag past b11618 existed on 2026-10-14, so `JLLAMA_GIT_TAG b11632` is paired with `JLLAMA_GIT_COMMIT 62df1815b…` and the production path ran exactly as CI will -- `FetchContent` by commit, the applier (stamp head `62df1815b`), and the pin check reporting `fetched commit count 11632 = b11632` (`git rev-list --count`, the number upstream's `release.yml` stamps the tag with and the one llama.cpp's own `build-info` reports). Then the static test build, `ctest` **601/601** (608 minus the seven of `test_model_split.cpp`), and the model-free native Java tests over the library (`NativeLibraryLoadSmokeTest` with the `LlamaCppVersion` pin `b11632` against the linked `b11632-62df1815b`, `LlamaLoggerTest`, `RpcServerTest`, `BackendLoadTest`: 27/27). The same chain had run at `df18bee11` / b11626 first (601/601, 27/27, `b11626-df18bee11`). The other drop-checks against pristine `62df1815b`: no `kolibri` (`0016`), no `common_log` hook (`0014`), no `stop_server` (`0015`), uncast `_mm_prefetch` (`0017`), `free(buffer->context)` (`0018`), the software fp16 conversion on `__ARM_NEON` (`0019`), the `GetCommandLineW` override and no `common_params_parse_main` (`0001`), the unconditional load-progress assignment (`0002`), no `LLAMA_SERVER_WORKER_CMD` (`0008`), no `get_slot_prompt_similarity` (`0003`). `check-natives.py`: 23 natives jars, 0 disagreements; spotless clean. Not run locally: the model-backed Java suite and the modular build -- CI covers both. When upstream publishes `b11632`, clear `JLLAMA_GIT_COMMIT` (`llama-next-version.sh` prints the NOTE); nothing else moves. | +| b11632–b11639 | 7 commits on master's head of 2026-10-14 (`a0e6dc738` = b11639, pinned through `JLLAMA_GIT_COMMIT` ahead of the tag -- upstream's newest tag is b11627), 21 files, ~691/90 lines (57 KiB; no `tools/ui`, no `.github/workflows`, no `common/`, `include/`, `src/`, `tools/`, root `CMakeLists.txt` unchanged, so the `_DEFAULT` re-check of "Modular natives" has nothing new). `ggml/src/ggml-cpu/ggml-cpu.c` (#30454, b11633): `GGML_OP_SET_ROWS` moves to the `n_tasks = n_threads` group -- every thread ran it with its own row of the F16 -> F32 conversion buffer while the planner sized the work buffer for one, so threads > 0 wrote past it whenever the graph held another multi-threaded op; every CPU module rebuilds it. `ggml/src/ggml-cpu/simd-mappings.h` (#30343, b11635): x86 with F16C keeps the lookup table for FP16 -> FP32 (clang lowers the F16C intrinsic into the dot products' live accumulator and serialises the loop, 45-65 % slower Q8_0/Q4_0 `MUL_MAT_ID`; the Windows x86-64 jar is built with plain clang) -- the carve-out of `0019`, see the patches row. `ggml-sycl` (#29107, b11636): persistent IQ3_S/IQ3_XXS reorder **and a build change: `GGML_SYCL_DNN` (default ON) now fails the configure when oneDNN is missing or CPU-only** instead of building without it -- `build-linux-x86_64-sycl` installed only DPC++ + MKL (the shipped `libggml-sycl.so` of the b11558 snapshot has no `libdnnl` in its `DT_NEEDED`), so the job now also installs `intel-oneapi-dnnl-devel`; the Windows job already installed `intel.oneapi.win.dnnl` and its module imports `dnnl.dll`. `ggml-opencl` (#30186, b11639): a strided CLAMP kernel for non-contiguous views (the OpenCL module jars). Docs, `gguf-py` and `.devops` Dockerfiles otherwise. **No project source change; one workflow change (the oneDNN package).** | +| b11632–b11639 | patches + upstream verification | **All twelve patches apply unchanged** (`git apply` in order on a pristine `a0e6dc738` worktree; `check-patches.py`: 12 patches, 94 hunks, 0 problems). **`0019`'s `&& !defined(__F16C__)` carve-out is load-bearing from this pin on**, and holds: `gcc -E` of the patched `simd-mappings.h` is silent for `-march=x86-64`, `-msse4.2`, `-march=haswell`, `-march=sapphirerapids`, `-march=znver4` and `-mavx -mf16c`. Drop-checks against pristine b11639 are those of the b11632 row, the range touching no other patch-target file: no `kolibri` (`0016`), no `common_log` hook (`0014`), no `stop_server` (`0015`), uncast `_mm_prefetch` (`0017`), `free(buffer->context)` (`0018`), the software fp32 -> fp16 conversion on `__ARM_NEON` (`0019`, #30505 still open), the `GetCommandLineW` override and no `common_params_parse_main` (`0001`), the unconditional load-progress assignment (`0002`), no `LLAMA_SERVER_WORKER_CMD` (`0008`), no `get_slot_prompt_similarity` (`0003`). Through the production path -- `rm -rf build`, `FetchContent` by commit, the applier, the pin check reporting `fetched commit count 11639 = b11639` -- the static test build, `ctest` **601/601**, and the model-free native Java tests over the library (`NativeLibraryLoadSmokeTest` with the `LlamaCppVersion` pin `b11639` against the linked build, `LlamaLoggerTest`, `RpcServerTest`, `BackendLoadTest`: 27/27). `check-natives.py`: 23 natives jars, 0 disagreements. Not run locally: the model-backed Java suite, the modular build and the SYCL job with oneDNN -- CI covers them. | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index a7ee88b0..0e1e1b58 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -344,8 +344,8 @@ endif() # hash, every other consumer keeps the number, and the pin check after FetchContent_MakeAvailable # asserts that the fetched commit's count IS the number. Clear it once the tag exists; # `llama-next-version.sh` says when, and `llama-next-version.sh --resolve b` prints the hash. -set(JLLAMA_GIT_TAG b11632) -set(JLLAMA_GIT_COMMIT "62df1815bc053066761f4eac58e288f23778e164") # until upstream tags b11632 +set(JLLAMA_GIT_TAG b11639) +set(JLLAMA_GIT_COMMIT "a0e6dc7384edb8f5d927155e1d26c10b4a310e6d") # until upstream tags b11639 if(NOT JLLAMA_GIT_TAG MATCHES "^b[0-9]+$") message(FATAL_ERROR "JLLAMA_GIT_TAG must be a llama.cpp release number b, got '${JLLAMA_GIT_TAG}'") endif() diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index c7e99072..519fc3ae 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -9,13 +9,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11632"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11639"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b11632-"} — call + * plus the resolved upstream commit, e.g. {@code "b11639-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -23,14 +23,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11632"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11639"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b11632"; + public static final String LLAMA_CPP_VERSION = "b11639"; // Constants holder — not instantiable. private LlamaCppVersion() {}