From da7fdd54377e395b1deb17517a57f0b915524535 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 8 Oct 2026 18:14:40 +0000 Subject: [PATCH 1/8] Docs: the CPU variants ship with 5.2.0, not 5.3.0 The reactor is at 5.2.0-SNAPSHOT (last release v5.1.0), so the first release with the Linux CPU-variant builds is 5.2.0; CLAUDE.md and TODO.md named 5.3.0. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2 --- CLAUDE.md | 2 +- TODO.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index f8a424aed..0132e6f59 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -77,7 +77,7 @@ step, not by Maven. ## CPU variants (Linux x86-64 and aarch64): `JLLAMA_CPU_VARIANTS` -The two Linux CPU jars, `cpu-linux-x86-64` and `cpu-linux-aarch64`, are built since 5.3.0 the way +The two Linux CPU jars, `cpu-linux-x86-64` and `cpu-linux-aarch64`, are built since 5.2.0 the way upstream builds its own release binaries: `GGML_BACKEND_DL` + `GGML_CPU_ALL_VARIANTS`, one small `ggml-cpu-` module per instruction-set level, of which `ggml_backend_load_best` loads the best at run time (it scores each module's own feature check against the running CPU). Before, each jar diff --git a/TODO.md b/TODO.md index 4436089ad..137444578 100644 --- a/TODO.md +++ b/TODO.md @@ -19,7 +19,7 @@ so everything below is genuinely still open. ### CPU variants (`JLLAMA_CPU_VARIANTS`) -- Windows, GPU modules, loader follow-ups -Linux x86-64 and aarch64 ship the variant build since 5.3.0 (CLAUDE.md "CPU variants"). The rest, +Linux x86-64 and aarch64 ship the variant build since 5.2.0 (CLAUDE.md "CPU variants"). The rest, with what a measurement on a Windows 11 machine (Ryzen 7 5800H, RTX 3070, JDK 21, upstream b11476 binaries; 2026-10-08) established: From c7b8cce6d35b4578481d4d7c1e309272dc553b9d Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 8 Oct 2026 19:09:44 +0000 Subject: [PATCH 2/8] Upgrade llama.cpp from b11476 to b11480 #29876 hands the chat parser the generated tokens next to the text: common_chat_input replaces the std::string of common_chat_parse(), of common_chat_parser_params::generation_prompt and of the server result types' content. Its string constructor is explicit, so the C++ tests wrap their literals; the wire format is unchanged. #29887 adds a GPU cache for the MoE experts a --cpu-moe setup keeps in host memory (--moe-cache-mib, EXPERIMENTAL). Exposed as ModelParameters.setMoeCacheMib(int) with ModelOption.MOE_CACHE_MIB, negative values rejected like upstream's parser does. All ten patches apply unchanged (verified in order against the pristine tag and by a fresh configure); see docs/history/llama-cpp-breaking-changes.md for the range. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2 --- CHANGELOG.md | 3 +++ CLAUDE.md | 8 +++---- README.md | 2 +- docs/history/llama-cpp-breaking-changes.md | 2 ++ llama/CMakeLists.txt | 2 +- .../net/ladenthin/llama/args/ModelOption.java | 3 +++ .../llama/parameters/ModelParameters.java | 21 ++++++++++++++++++ .../llama/value/LlamaCppVersion.java | 8 +++---- llama/src/test/cpp/test_server.cpp | 22 +++++++++---------- llama/src/test/cpp/test_utils.cpp | 2 +- .../llama/parameters/ModelParametersTest.java | 21 ++++++++++++++++++ 11 files changed, 72 insertions(+), 22 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 64c59759c..6cc3a6813 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,9 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by ## [Unreleased] ### Added +- **`ModelParameters.setMoeCacheMib(int)`** (`--moe-cache-mib`, llama.cpp b11480, EXPERIMENTAL upstream): a + GPU cache for the MoE expert weights a `setCpuMoeLayers` / `--cpu-moe` setup keeps in host memory, + split among several GPUs like the layers; `0` (the default) disables it. - **Kolibri-1 support** (Aleph Alpha, architecture `kolibri1`, 78B German/English reasoning MoE) ahead of upstream llama.cpp ([ggml-org/llama.cpp#29922](https://github.com/ggml-org/llama.cpp/issues/29922)), as the carried patch `0016-model-kolibri1.patch`. It combines the two community ports and, unlike either of them, loads the GGUFs of diff --git a/CLAUDE.md b/CLAUDE.md index 0132e6f59..98594ffd5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b11476** +Current llama.cpp pinned version: **b11480** ## Natives jars: one directory per backend (`.github/natives.csv`) @@ -787,7 +787,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network for the asset build; the embed step is plain cmake -P -git clone --depth 1 --branch b11476 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b11480 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build ) mkdir -p webui-generated /tmp/ui-gen cmake -DUI_SOURCE_DIR=/tmp/lc/tools/ui -DUI_BINARY_DIR=/tmp/ui-gen \ @@ -827,7 +827,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b11476`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b11480`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -1966,7 +1966,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b11476`. +llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b11480`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.18.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index 9a4d3b262..b4c848dd1 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b11476](https://img.shields.io/badge/llama.cpp-%23b11476-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11476) +[![llama.cpp b11480](https://img.shields.io/badge/llama.cpp-%23b11480-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11480) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index abb5e2b05..f116a1d2f 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -836,3 +836,5 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b11462–b11474 | patches + upstream verification | **All ten patches apply unchanged**, verified by a fresh configure through the real `FetchContent` path. Two patch-target files are in the range -- `src/llama-vocab.cpp` and `src/models/models.h`, both touched by `0016` -- but the upstream hunks (the FIM token lists and the GLM5-Next structs) are away from `0016`'s, so only line offsets moved. The drop-checks hold: nothing under `tools/server/`, `include/`, `common/arg.cpp`, `common/log.{h,cpp}`, `src/llama-model.cpp` or `ggml/src/ggml-rpc/` changed, and `git grep -i kolibri` finds nothing under `src/` at b11474. No `.github/` change upstream: OpenVINO, CCCL (`v3.4.3`) and the ROCm wheels stay where b11457 left them. | | b11474–b11476 | Two commits, 4 files, ~60/5 lines (8 KiB, one step), Metal only. #30100 fixes the `MUL_MAT`+`ADD` fusion when the residual is itself a `MUL_MAT` output: `ggml_metal_op_mul_mat_mma` picked the residual as "the operand that is not a `MUL_MAT`", which chose the wrong one when both are; it now takes the other operand of the `ADD` by identity. #30065 extends the few-row MMA mat-mul (`mul_mv_mma`) to the remaining `src0` types (BF16, the K-quants Q2_K/Q3_K, MXFP4, the IQ and TQ types), with per-type minimum row counts. Plus `test-backend-ops` cases (not built here). Only the shipped `metal-macos-aarch64` build and the two test-only macOS Metal builds compile it. **No project source change.** This is the target of the bump. | | b11474–b11476 | patches + upstream verification | **All ten patches apply unchanged**, verified in order against pristine b11476 with `git apply`. Nothing outside `ggml/src/ggml-metal/` and `tests/` changed, so no patch-target file is in the range and every drop-check of the b11474 row holds unchanged (`0016` included: still no `kolibri` under `src/`). No `.github/` change upstream. | +| b11476–b11480 | Four commits, 43 files, ~1068/199 lines (94 KiB, one step). **#29876 hands the chat parser text *and* tokens.** `common_chat_input` (`common/chat.h`: the text plus one token id per byte, `LLAMA_TOKEN_NULL` where a byte has none; `append`/`prepend`/`truncate`/`substr`, and `common_chat_input_tokenize()`, which re-tokenizes a text and keeps the ids only when the pieces reproduce it) replaces the `std::string` of `common_chat_parse()` / `common_chat_peg_parse()`, of `common_chat_parser_params::generation_prompt` (the schema tokenizes it against the vocab now), of the slot's `generated` and of `server_task_result_cmpl_{partial,final}::content`; the PEG parse context carries the tokens (`peg-parser.h`), so a parser can match token ids instead of text. Every `to_json()` emits `content.text` -- request fields, bounds and response keys are identical between the tags. **Project change:** the `std::string` constructor is `explicit`, so `test_server.cpp`'s ten `content = "..."` assignments, its one `generation_prompt = "..."` and `test_utils.cpp`'s `common_chat_parse(raw, ...)` wrap the string in `common_chat_input(...)`; `jllama.cpp` reads results through `to_json()` only. **#29887 adds a GPU cache for the MoE experts kept in host memory**: `llama_context_params.moe_cache_size` (after `type_v`, EXPERIMENTAL), `common_params.moe_cache_size`, the CLI option `--moe-cache-mib N` (env `LLAMA_ARG_MOE_CACHE_MIB`, negatives rejected; the draft context gets 0) and the new `src/llama-moe-cache.{cpp,h}`: experts a `--n-cpu-moe` / `--cpu-moe` setup leaves in RAM are cached on the device up to that budget. Exposed as **`ModelParameters.setMoeCacheMib(int)`** (`ModelOption.MOE_CACHE_MIB`). #30097 moves the trunk-only / MTP-only tensor flags the NextN models each copied into a `nextn_flags` helper of `llama_model_base` (six more architectures accept a trunk-only file); #29425 CUDA `pool2d`. Plus `test-chat.cpp`, `test-llama-archs.cpp` and `test-backend-ops.cpp`, not built here. | +| b11476–b11480 | patches + upstream verification | **All ten patches apply unchanged**, verified in order against pristine b11480 with `git apply`, then by a fresh configure through the real `FetchContent` path. Patch-target files in the range: `common/arg.cpp` (`0001`, `0015`: the new `--moe-cache-mib` option is away from `common_params_parse` and `add_rpc_devices`), `tools/server/server-context.cpp` (`0002`, `0003`: the `generated` rename is away from `load_model` and the similarity getter/setter) and `src/llama-model.{cpp,h}` (`0012`, `0016`: the `nextn_flags` helper is away from `load_tensors`' split block and the Kolibri-1 registrations). Drop-checks against pristine b11480 unchanged: `0001` override present and `common_params_parse_main` absent, `0002` assignment unconditional, no `0003` getter, no `0006`-`0008` counterpart, no `0012` guard, no `0014` hook (`common/log.h` still names only `common_log_default_callback`), no `0015` `stop_server`, no `kolibri` under `src/`. No `.github/` change upstream anywhere in b11476..b11512: CCCL `v3.4.3`, OpenVINO 2026.4.1 and ROCm 10.0.0 stay. | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index 9b992e80b..300a73b69 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -216,7 +216,7 @@ endif() FetchContent_Declare( llama.cpp GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git - GIT_TAG b11476 + GIT_TAG b11480 PATCH_COMMAND ${CMAKE_COMMAND} -DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches -DLLAMA_SRC= diff --git a/llama/src/main/java/net/ladenthin/llama/args/ModelOption.java b/llama/src/main/java/net/ladenthin/llama/args/ModelOption.java index 145b2b3d0..729cbc696 100644 --- a/llama/src/main/java/net/ladenthin/llama/args/ModelOption.java +++ b/llama/src/main/java/net/ladenthin/llama/args/ModelOption.java @@ -191,6 +191,9 @@ public enum ModelOption { /** CLI option {@code --model-url}. */ MODEL_URL("--model-url"), + /** CLI option {@code --moe-cache-mib}. */ + MOE_CACHE_MIB("--moe-cache-mib"), + /** CLI option {@code --n-cpu-ffn}. */ N_CPU_FFN("--n-cpu-ffn"), diff --git a/llama/src/main/java/net/ladenthin/llama/parameters/ModelParameters.java b/llama/src/main/java/net/ladenthin/llama/parameters/ModelParameters.java index dd8541cd2..8f46103b9 100644 --- a/llama/src/main/java/net/ladenthin/llama/parameters/ModelParameters.java +++ b/llama/src/main/java/net/ladenthin/llama/parameters/ModelParameters.java @@ -909,6 +909,27 @@ public ModelParameters setCpuMoeLayers(int layers) { return putScalar(ModelOption.N_CPU_MOE, layers); } + /** + * Give the MoE expert weights kept in host memory a cache on the GPU of {@code mib} MiB. + * + *

Experts a {@link #setCpuMoeLayers(int)} / {@code --cpu-moe} setup leaves in RAM are + * otherwise read from host memory on every use. With a cache, llama.cpp keeps the most recently + * used experts on the device up to this budget; with several GPUs the budget is split among + * them like the layers ({@link #setTensorSplit(String)}). Upstream {@code --moe-cache-mib} + * (llama.cpp b11480, marked EXPERIMENTAL there); {@code 0}, the default, disables the cache. + * Without a GPU backend the option has no effect.

+ * + * @param mib the cache size in MiB, {@code 0} to disable + * @return this builder + * @throws IllegalArgumentException if {@code mib} is negative (upstream rejects it as well) + */ + public ModelParameters setMoeCacheMib(int mib) { + if (mib < 0) { + throw new IllegalArgumentException("Invalid moe-cache-mib value: " + mib + " (must be >= 0; 0 = disabled)"); + } + return putScalar(ModelOption.MOE_CACHE_MIB, mib); + } + /** * Keep the dense FFN weights of the first {@code n} layers on the CPU. * diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index 353a820bc..cf60c06fb 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -9,13 +9,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11476"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11480"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b11476-"} — call + * plus the resolved upstream commit, e.g. {@code "b11480-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -23,14 +23,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11476"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11480"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b11476"; + public static final String LLAMA_CPP_VERSION = "b11480"; // Constants holder — not instantiable. private LlamaCppVersion() {} diff --git a/llama/src/test/cpp/test_server.cpp b/llama/src/test/cpp/test_server.cpp index da59edba2..5a194c325 100644 --- a/llama/src/test/cpp/test_server.cpp +++ b/llama/src/test/cpp/test_server.cpp @@ -250,7 +250,7 @@ TEST(SlotParamsToJson, GrammarValue_OutputFormatGrammarExtracted) { TEST(SlotParamsToJson, GenerationPrompt_ReflectsSyntaxField) { task_params p; - p.chat_parser_params.generation_prompt = "Think step by step:"; + p.chat_parser_params.generation_prompt = common_chat_input("Think step by step:"); const json j = p.to_json(); @@ -1090,7 +1090,7 @@ TEST(ServerTaskResultCmplPartial, NonOaicompat_CoreFields) { server_task_result_cmpl_partial p; p.is_updated = true; p.res_type = TASK_RESPONSE_TYPE_NONE; - p.content = "hello"; + p.content = common_chat_input("hello"); p.n_decoded = 3; p.n_prompt_tokens = 10; @@ -1192,7 +1192,7 @@ TEST(ServerTaskResultCmplFinal, IsStop_ReturnsTrue) { TEST(ServerTaskResultCmplFinal, NonOaicompat_StopAlwaysTrue) { server_task_result_cmpl_final f; - f.content = "done"; + f.content = common_chat_input("done"); f.n_decoded = 3; f.n_prompt_tokens = 7; const json j = f.to_json_non_oaicompat(); @@ -1307,7 +1307,7 @@ TEST(ServerTaskResultCmplFinal, UsageJsonOaicompat_TotalTokensIsSumOfBoth) { namespace { server_task_result_cmpl_final make_oai_final(const std::string &content = "hello") { server_task_result_cmpl_final f; - f.content = content; + f.content = common_chat_input(content); f.oaicompat_model = "test-model"; f.oaicompat_cmpl_id = "cmpl-test"; f.n_decoded = 3; @@ -1575,7 +1575,7 @@ server_task_result_cmpl_partial make_partial(const std::string &content = "tok") server_task_result_cmpl_partial p; p.is_updated = true; p.res_type = TASK_RESPONSE_TYPE_OAI_CMPL; - p.content = content; + p.content = common_chat_input(content); p.oaicompat_model = "test-model"; p.oaicompat_cmpl_id = "cmpl-part"; return p; @@ -1636,7 +1636,7 @@ TEST(CmplPartialToJsonDispatch, ResTypeNone_RoutesToNonOaicompat) { server_task_result_cmpl_partial p; p.is_updated = true; p.res_type = TASK_RESPONSE_TYPE_NONE; - p.content = "hello"; + p.content = common_chat_input("hello"); const json j = p.to_json(); // must not assert/abort // non-oaicompat shape has "content" directly EXPECT_EQ(j.at("content").get(), "hello"); @@ -1646,7 +1646,7 @@ TEST(CmplPartialToJsonDispatch, ResTypeOaiCmpl_RoutesToOaicompat) { server_task_result_cmpl_partial p; p.is_updated = true; p.res_type = TASK_RESPONSE_TYPE_OAI_CMPL; - p.content = "hi"; + p.content = common_chat_input("hi"); p.oaicompat_model = "m"; p.oaicompat_cmpl_id = "c"; const json j = p.to_json(); @@ -1695,7 +1695,7 @@ server_task_result_cmpl_final make_dispatched_final(task_response_type rt, bool f.is_updated = true; f.res_type = rt; f.stream = stream; - f.content = "hi"; + f.content = common_chat_input("hi"); f.oaicompat_model = "m"; f.oaicompat_cmpl_id = "id"; return f; @@ -2108,7 +2108,7 @@ TEST(ParamsFromJsonCmpl, PlainGrammar_NoGrammarType_SetsUserType) { TEST(CmplFinalResponseFields, EmptyList_AllFieldsPresent) { server_task_result_cmpl_final f; - f.content = "hi"; + f.content = common_chat_input("hi"); f.stop = STOP_TYPE_EOS; // response_fields is empty by default → full object returned const json j = f.to_json_non_oaicompat(); @@ -2119,7 +2119,7 @@ TEST(CmplFinalResponseFields, EmptyList_AllFieldsPresent) { TEST(CmplFinalResponseFields, NonEmptyList_OnlyRequestedFieldsPresent) { server_task_result_cmpl_final f; - f.content = "projected"; + f.content = common_chat_input("projected"); f.response_fields = {"content", "tokens_predicted"}; const json j = f.to_json_non_oaicompat(); EXPECT_TRUE(j.contains("content")); @@ -2131,7 +2131,7 @@ TEST(CmplFinalResponseFields, NonEmptyList_OnlyRequestedFieldsPresent) { TEST(CmplFinalResponseFields, ContentValue_PreservedThroughProjection) { server_task_result_cmpl_final f; - f.content = "keep this"; + f.content = common_chat_input("keep this"); f.response_fields = {"content"}; const json j = f.to_json_non_oaicompat(); EXPECT_EQ(j.at("content").get(), "keep this"); diff --git a/llama/src/test/cpp/test_utils.cpp b/llama/src/test/cpp/test_utils.cpp index 135e25b7d..4cda471a7 100644 --- a/llama/src/test/cpp/test_utils.cpp +++ b/llama/src/test/cpp/test_utils.cpp @@ -1466,7 +1466,7 @@ namespace { // is_partial = false. std::string parse_content_only(const std::string &raw) { const common_chat_parser_params params; - return common_chat_parse(raw, /*is_partial=*/false, params).content; + return common_chat_parse(common_chat_input(raw), /*is_partial=*/false, params).content; } // U+FFFD REPLACEMENT CHARACTER, the byte sequence upstream substitutes for diff --git a/llama/src/test/java/net/ladenthin/llama/parameters/ModelParametersTest.java b/llama/src/test/java/net/ladenthin/llama/parameters/ModelParametersTest.java index 8500f4fcc..2b080a6f0 100644 --- a/llama/src/test/java/net/ladenthin/llama/parameters/ModelParametersTest.java +++ b/llama/src/test/java/net/ladenthin/llama/parameters/ModelParametersTest.java @@ -781,4 +781,25 @@ public void testSetDraftSamplingProbabilistic() { ModelParameters p = new ModelParameters().setDraftSampling(DraftSampling.PROBABILISTIC); assertThat(p.parameters.get("--spec-draft-sampling"), is("probabilistic")); } + + // ------------------------------------------------------------------------- + // setMoeCacheMib (llama.cpp b11480) + // ------------------------------------------------------------------------- + + @Test + public void testSetMoeCacheMib() { + ModelParameters p = new ModelParameters().setMoeCacheMib(2048); + assertThat(p.parameters.get("--moe-cache-mib"), is("2048")); + } + + @Test + public void testSetMoeCacheMibZeroDisables() { + ModelParameters p = new ModelParameters().setMoeCacheMib(0); + assertThat(p.parameters.get("--moe-cache-mib"), is("0")); + } + + @Test + public void testSetMoeCacheMibRejectsNegative() { + assertThrows(IllegalArgumentException.class, () -> new ModelParameters().setMoeCacheMib(-1)); + } } From 3398b7d23af4451686702ddea0e0adccb1e290fb Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 8 Oct 2026 19:10:17 +0000 Subject: [PATCH 3/8] Upgrade llama.cpp from b11480 to b11484 All ten patches apply unchanged (verified in order against the pristine tag); see docs/history/llama-cpp-breaking-changes.md for the range. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2 --- CLAUDE.md | 8 ++++---- README.md | 2 +- docs/history/llama-cpp-breaking-changes.md | 2 ++ llama/CMakeLists.txt | 2 +- .../java/net/ladenthin/llama/value/LlamaCppVersion.java | 8 ++++---- 5 files changed, 12 insertions(+), 10 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 98594ffd5..dcc55e840 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b11480** +Current llama.cpp pinned version: **b11484** ## Natives jars: one directory per backend (`.github/natives.csv`) @@ -787,7 +787,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network for the asset build; the embed step is plain cmake -P -git clone --depth 1 --branch b11480 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b11484 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build ) mkdir -p webui-generated /tmp/ui-gen cmake -DUI_SOURCE_DIR=/tmp/lc/tools/ui -DUI_BINARY_DIR=/tmp/ui-gen \ @@ -827,7 +827,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b11480`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b11484`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -1966,7 +1966,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b11480`. +llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b11484`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.18.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index b4c848dd1..43e6d3d7d 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b11480](https://img.shields.io/badge/llama.cpp-%23b11480-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11480) +[![llama.cpp b11484](https://img.shields.io/badge/llama.cpp-%23b11484-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11484) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index f116a1d2f..8dae6d5fc 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -838,3 +838,5 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b11474–b11476 | patches + upstream verification | **All ten patches apply unchanged**, verified in order against pristine b11476 with `git apply`. Nothing outside `ggml/src/ggml-metal/` and `tests/` changed, so no patch-target file is in the range and every drop-check of the b11474 row holds unchanged (`0016` included: still no `kolibri` under `src/`). No `.github/` change upstream. | | b11476–b11480 | Four commits, 43 files, ~1068/199 lines (94 KiB, one step). **#29876 hands the chat parser text *and* tokens.** `common_chat_input` (`common/chat.h`: the text plus one token id per byte, `LLAMA_TOKEN_NULL` where a byte has none; `append`/`prepend`/`truncate`/`substr`, and `common_chat_input_tokenize()`, which re-tokenizes a text and keeps the ids only when the pieces reproduce it) replaces the `std::string` of `common_chat_parse()` / `common_chat_peg_parse()`, of `common_chat_parser_params::generation_prompt` (the schema tokenizes it against the vocab now), of the slot's `generated` and of `server_task_result_cmpl_{partial,final}::content`; the PEG parse context carries the tokens (`peg-parser.h`), so a parser can match token ids instead of text. Every `to_json()` emits `content.text` -- request fields, bounds and response keys are identical between the tags. **Project change:** the `std::string` constructor is `explicit`, so `test_server.cpp`'s ten `content = "..."` assignments, its one `generation_prompt = "..."` and `test_utils.cpp`'s `common_chat_parse(raw, ...)` wrap the string in `common_chat_input(...)`; `jllama.cpp` reads results through `to_json()` only. **#29887 adds a GPU cache for the MoE experts kept in host memory**: `llama_context_params.moe_cache_size` (after `type_v`, EXPERIMENTAL), `common_params.moe_cache_size`, the CLI option `--moe-cache-mib N` (env `LLAMA_ARG_MOE_CACHE_MIB`, negatives rejected; the draft context gets 0) and the new `src/llama-moe-cache.{cpp,h}`: experts a `--n-cpu-moe` / `--cpu-moe` setup leaves in RAM are cached on the device up to that budget. Exposed as **`ModelParameters.setMoeCacheMib(int)`** (`ModelOption.MOE_CACHE_MIB`). #30097 moves the trunk-only / MTP-only tensor flags the NextN models each copied into a `nextn_flags` helper of `llama_model_base` (six more architectures accept a trunk-only file); #29425 CUDA `pool2d`. Plus `test-chat.cpp`, `test-llama-archs.cpp` and `test-backend-ops.cpp`, not built here. | | b11476–b11480 | patches + upstream verification | **All ten patches apply unchanged**, verified in order against pristine b11480 with `git apply`, then by a fresh configure through the real `FetchContent` path. Patch-target files in the range: `common/arg.cpp` (`0001`, `0015`: the new `--moe-cache-mib` option is away from `common_params_parse` and `add_rpc_devices`), `tools/server/server-context.cpp` (`0002`, `0003`: the `generated` rename is away from `load_model` and the similarity getter/setter) and `src/llama-model.{cpp,h}` (`0012`, `0016`: the `nextn_flags` helper is away from `load_tensors`' split block and the Kolibri-1 registrations). Drop-checks against pristine b11480 unchanged: `0001` override present and `common_params_parse_main` absent, `0002` assignment unconditional, no `0003` getter, no `0006`-`0008` counterpart, no `0012` guard, no `0014` hook (`common/log.h` still names only `common_log_default_callback`), no `0015` `stop_server`, no `kolibri` under `src/`. No `.github/` change upstream anywhere in b11476..b11512: CCCL `v3.4.3`, OpenVINO 2026.4.1 and ROCm 10.0.0 stay. | +| b11480–b11484 | Four commits, 42 files, ~635/106 lines (95 KiB, one step). **#30088 names the tool and argument rules of the generated PEG parsers by index** instead of by the tool's and the arguments' names (`common/parsers`, 17 files, `chat-auto-parser-generator.cpp`, `chat-peg-parser.cpp`): a tool named `call` collided with a built-in rule and sent the parser into infinite recursion (#29967). The grammars the server emits are otherwise the same; `test_server.cpp`'s grammar assertions are literal user grammars and untouched, and tool calling through `ToolDefinition` / the OpenAI-compatible server stays covered by `ToolCallingIntegrationTest`. **#30110 adds the LiquidAI d1-3B decision model**: `COMMON_DECISION_TYPE_LFM2_D1` (`"lfm2-d1"`, openjev-style labels that depend on the question type), `server_task::decision::label_groups` (several label tokens per output, the output is their maximum -- reduced in `server-context.cpp` right after the logits are read), `server-decision.{h,cpp}`, and mtmd reads the LFM2 image-resize algorithm from the GGUF. `LlamaModel.handleSystemOne` and `ModelMeta.isDecisionModel()` (output modalities `["decisions"]`) serve it unchanged. #30062 adds the cohere2 vision projector to mtmd, #29100 CUDA FWHT kernels for block widths 1024-8192. **No project source change.** | +| b11480–b11484 | patches + upstream verification | **All ten patches apply unchanged**, verified in order against pristine b11484 with `git apply`. Patch-target file in the range: `tools/server/server-context.cpp` (`0002`, `0003`: the `label_groups` reduction sits in the decision result path, away from `load_model` and the similarity getter/setter). Nothing under `common/arg.cpp`, `common/log.h`, `src/llama-model.cpp` or `ggml-rpc` changed, so every drop-check of the b11480 row holds unchanged. | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index 300a73b69..ecfdead8a 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -216,7 +216,7 @@ endif() FetchContent_Declare( llama.cpp GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git - GIT_TAG b11480 + GIT_TAG b11484 PATCH_COMMAND ${CMAKE_COMMAND} -DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches -DLLAMA_SRC= diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index cf60c06fb..4a8d0ca6a 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -9,13 +9,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11480"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11484"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b11480-"} — call + * plus the resolved upstream commit, e.g. {@code "b11484-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -23,14 +23,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11480"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11484"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b11480"; + public static final String LLAMA_CPP_VERSION = "b11484"; // Constants holder — not instantiable. private LlamaCppVersion() {} From e075e725b8226aa4e19aa6c102e4cedbfbbb70f7 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 8 Oct 2026 19:10:17 +0000 Subject: [PATCH 4/8] Upgrade llama.cpp from b11484 to b11487 All ten patches apply unchanged (verified in order against the pristine tag); see docs/history/llama-cpp-breaking-changes.md for the range. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2 --- CLAUDE.md | 8 ++++---- README.md | 2 +- docs/history/llama-cpp-breaking-changes.md | 2 ++ llama/CMakeLists.txt | 2 +- .../java/net/ladenthin/llama/value/LlamaCppVersion.java | 8 ++++---- 5 files changed, 12 insertions(+), 10 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index dcc55e840..884be5775 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b11484** +Current llama.cpp pinned version: **b11487** ## Natives jars: one directory per backend (`.github/natives.csv`) @@ -787,7 +787,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network for the asset build; the embed step is plain cmake -P -git clone --depth 1 --branch b11484 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b11487 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build ) mkdir -p webui-generated /tmp/ui-gen cmake -DUI_SOURCE_DIR=/tmp/lc/tools/ui -DUI_BINARY_DIR=/tmp/ui-gen \ @@ -827,7 +827,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b11484`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b11487`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -1966,7 +1966,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b11484`. +llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b11487`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.18.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index 43e6d3d7d..01878bc90 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b11484](https://img.shields.io/badge/llama.cpp-%23b11484-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11484) +[![llama.cpp b11487](https://img.shields.io/badge/llama.cpp-%23b11487-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11487) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index 8dae6d5fc..5aced3253 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -840,3 +840,5 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b11476–b11480 | patches + upstream verification | **All ten patches apply unchanged**, verified in order against pristine b11480 with `git apply`, then by a fresh configure through the real `FetchContent` path. Patch-target files in the range: `common/arg.cpp` (`0001`, `0015`: the new `--moe-cache-mib` option is away from `common_params_parse` and `add_rpc_devices`), `tools/server/server-context.cpp` (`0002`, `0003`: the `generated` rename is away from `load_model` and the similarity getter/setter) and `src/llama-model.{cpp,h}` (`0012`, `0016`: the `nextn_flags` helper is away from `load_tensors`' split block and the Kolibri-1 registrations). Drop-checks against pristine b11480 unchanged: `0001` override present and `common_params_parse_main` absent, `0002` assignment unconditional, no `0003` getter, no `0006`-`0008` counterpart, no `0012` guard, no `0014` hook (`common/log.h` still names only `common_log_default_callback`), no `0015` `stop_server`, no `kolibri` under `src/`. No `.github/` change upstream anywhere in b11476..b11512: CCCL `v3.4.3`, OpenVINO 2026.4.1 and ROCm 10.0.0 stay. | | b11480–b11484 | Four commits, 42 files, ~635/106 lines (95 KiB, one step). **#30088 names the tool and argument rules of the generated PEG parsers by index** instead of by the tool's and the arguments' names (`common/parsers`, 17 files, `chat-auto-parser-generator.cpp`, `chat-peg-parser.cpp`): a tool named `call` collided with a built-in rule and sent the parser into infinite recursion (#29967). The grammars the server emits are otherwise the same; `test_server.cpp`'s grammar assertions are literal user grammars and untouched, and tool calling through `ToolDefinition` / the OpenAI-compatible server stays covered by `ToolCallingIntegrationTest`. **#30110 adds the LiquidAI d1-3B decision model**: `COMMON_DECISION_TYPE_LFM2_D1` (`"lfm2-d1"`, openjev-style labels that depend on the question type), `server_task::decision::label_groups` (several label tokens per output, the output is their maximum -- reduced in `server-context.cpp` right after the logits are read), `server-decision.{h,cpp}`, and mtmd reads the LFM2 image-resize algorithm from the GGUF. `LlamaModel.handleSystemOne` and `ModelMeta.isDecisionModel()` (output modalities `["decisions"]`) serve it unchanged. #30062 adds the cohere2 vision projector to mtmd, #29100 CUDA FWHT kernels for block widths 1024-8192. **No project source change.** | | b11480–b11484 | patches + upstream verification | **All ten patches apply unchanged**, verified in order against pristine b11484 with `git apply`. Patch-target file in the range: `tools/server/server-context.cpp` (`0002`, `0003`: the `label_groups` reduction sits in the decision result path, away from `load_model` and the similarity getter/setter). Nothing under `common/arg.cpp`, `common/log.h`, `src/llama-model.cpp` or `ggml-rpc` changed, so every drop-check of the b11480 row holds unchanged. | +| b11484–b11487 | Three commits, 14 files, ~919/27 lines (46 KiB, one step). **#30096 fixes the Jinja parser for TranslateGemma** (`common/chat.cpp`, three parser files, the template under `models/templates/`, `test-chat.cpp`) and logs a missing `source_lang_code` / `target_lang_code` instead of failing silently -- a chat-template fix that reaches the embedded server through `--jinja` without any project change. The rest is the Hexagon backend (tiled Q4_K/Q6_K `GET_ROWS` #30115, GELU accuracy #30104), which no natives jar builds. **No project source change.** | +| b11484–b11487 | patches + upstream verification | **All ten patches apply unchanged**, verified in order against pristine b11487 with `git apply`. No patch-target file in the range; nothing under `src/`, `tools/server/`, `common/arg.cpp`, `common/log.h` or `include/` changed, so every drop-check of the b11480 row holds unchanged. | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index ecfdead8a..4d3f9e795 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -216,7 +216,7 @@ endif() FetchContent_Declare( llama.cpp GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git - GIT_TAG b11484 + GIT_TAG b11487 PATCH_COMMAND ${CMAKE_COMMAND} -DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches -DLLAMA_SRC= diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index 4a8d0ca6a..6c135f537 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -9,13 +9,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11484"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11487"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b11484-"} — call + * plus the resolved upstream commit, e.g. {@code "b11487-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -23,14 +23,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11484"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11487"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b11484"; + public static final String LLAMA_CPP_VERSION = "b11487"; // Constants holder — not instantiable. private LlamaCppVersion() {} From 2e387365fea3d41c691ea2f7d6ab3c8be06e70f7 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 8 Oct 2026 19:10:17 +0000 Subject: [PATCH 5/8] Upgrade llama.cpp from b11487 to b11491 All ten patches apply unchanged (verified in order against the pristine tag); see docs/history/llama-cpp-breaking-changes.md for the range. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2 --- CLAUDE.md | 8 ++++---- README.md | 2 +- docs/history/llama-cpp-breaking-changes.md | 2 ++ llama/CMakeLists.txt | 2 +- .../java/net/ladenthin/llama/value/LlamaCppVersion.java | 8 ++++---- 5 files changed, 12 insertions(+), 10 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 884be5775..97ff69d69 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b11487** +Current llama.cpp pinned version: **b11491** ## Natives jars: one directory per backend (`.github/natives.csv`) @@ -787,7 +787,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network for the asset build; the embed step is plain cmake -P -git clone --depth 1 --branch b11487 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b11491 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build ) mkdir -p webui-generated /tmp/ui-gen cmake -DUI_SOURCE_DIR=/tmp/lc/tools/ui -DUI_BINARY_DIR=/tmp/ui-gen \ @@ -827,7 +827,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b11487`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b11491`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -1966,7 +1966,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b11487`. +llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b11491`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.18.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index 01878bc90..208ed55d2 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b11487](https://img.shields.io/badge/llama.cpp-%23b11487-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11487) +[![llama.cpp b11491](https://img.shields.io/badge/llama.cpp-%23b11491-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11491) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index 5aced3253..c45ca241f 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -842,3 +842,5 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b11480–b11484 | patches + upstream verification | **All ten patches apply unchanged**, verified in order against pristine b11484 with `git apply`. Patch-target file in the range: `tools/server/server-context.cpp` (`0002`, `0003`: the `label_groups` reduction sits in the decision result path, away from `load_model` and the similarity getter/setter). Nothing under `common/arg.cpp`, `common/log.h`, `src/llama-model.cpp` or `ggml-rpc` changed, so every drop-check of the b11480 row holds unchanged. | | b11484–b11487 | Three commits, 14 files, ~919/27 lines (46 KiB, one step). **#30096 fixes the Jinja parser for TranslateGemma** (`common/chat.cpp`, three parser files, the template under `models/templates/`, `test-chat.cpp`) and logs a missing `source_lang_code` / `target_lang_code` instead of failing silently -- a chat-template fix that reaches the embedded server through `--jinja` without any project change. The rest is the Hexagon backend (tiled Q4_K/Q6_K `GET_ROWS` #30115, GELU accuracy #30104), which no natives jar builds. **No project source change.** | | b11484–b11487 | patches + upstream verification | **All ten patches apply unchanged**, verified in order against pristine b11487 with `git apply`. No patch-target file in the range; nothing under `src/`, `tools/server/`, `common/arg.cpp`, `common/log.h` or `include/` changed, so every drop-check of the b11480 row holds unchanged. | +| b11487–b11491 | Four commits, 28 files, ~1104/70 lines (92 KiB, one step). **#30114 adds the LiquidAI d1-omni-600M decision model** (audio and image input): `COMMON_DECISION_TYPE_LFM2_D1_OMNI` (`"lfm2-d1-omni"`, laya-style with another prompt layout), a memory-less LFM2 graph for the non-causal case (`src/models/lfm2.cpp`, +295; `models.h`, `llama-model.cpp`), mtmd's conformer GLU sigmoid kept on CUDA and new `mtmd-audio` helpers, and in the server audio arrives through `images` / `input_audio`, with `images` now an alias of `files` on `/v1/systemone` (`server-decision.{h,cpp}`, +233; README). `LlamaModel.handleSystemOne` hands the request body through, so the alias and the audio input are available to Java callers as they are. #29507 removes duplicate SYCL block-size defines; #30126/#30121 Hexagon (`alloc_buffer_n`, Q6_K dequant). **No project source change.** | +| b11487–b11491 | patches + upstream verification | **All ten patches apply unchanged**, verified in order against pristine b11491 with `git apply`. Patch-target files in the range: `src/llama-model.cpp` and `src/models/models.h` (`0012`, `0016`: the LFM2 additions are away from `load_tensors`' split block and from the Kolibri-1 registration and struct) and `tools/server/server-context.cpp` (`0002`, `0003`; a two-line change in the decision path). Drop-checks unchanged: still no `split_sum` guard, still no `kolibri` under `src/`. | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index 4d3f9e795..22383897b 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -216,7 +216,7 @@ endif() FetchContent_Declare( llama.cpp GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git - GIT_TAG b11487 + GIT_TAG b11491 PATCH_COMMAND ${CMAKE_COMMAND} -DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches -DLLAMA_SRC= diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index 6c135f537..b0047b1ea 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -9,13 +9,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11487"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11491"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b11487-"} — call + * plus the resolved upstream commit, e.g. {@code "b11491-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -23,14 +23,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11487"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11491"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b11487"; + public static final String LLAMA_CPP_VERSION = "b11491"; // Constants holder — not instantiable. private LlamaCppVersion() {} From a6559ec01cf4740e587d57d4b669429a40aa34b3 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 8 Oct 2026 19:10:18 +0000 Subject: [PATCH 6/8] Upgrade llama.cpp from b11491 to b11495 All ten patches apply unchanged (verified in order against the pristine tag); see docs/history/llama-cpp-breaking-changes.md for the range. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2 --- CLAUDE.md | 8 ++++---- README.md | 2 +- docs/history/llama-cpp-breaking-changes.md | 2 ++ llama/CMakeLists.txt | 2 +- .../java/net/ladenthin/llama/value/LlamaCppVersion.java | 8 ++++---- 5 files changed, 12 insertions(+), 10 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 97ff69d69..5a566cb11 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b11491** +Current llama.cpp pinned version: **b11495** ## Natives jars: one directory per backend (`.github/natives.csv`) @@ -787,7 +787,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network for the asset build; the embed step is plain cmake -P -git clone --depth 1 --branch b11491 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b11495 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build ) mkdir -p webui-generated /tmp/ui-gen cmake -DUI_SOURCE_DIR=/tmp/lc/tools/ui -DUI_BINARY_DIR=/tmp/ui-gen \ @@ -827,7 +827,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b11491`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b11495`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -1966,7 +1966,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b11491`. +llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b11495`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.18.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index 208ed55d2..8e1482971 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b11491](https://img.shields.io/badge/llama.cpp-%23b11491-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11491) +[![llama.cpp b11495](https://img.shields.io/badge/llama.cpp-%23b11495-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11495) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index c45ca241f..428d167ab 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -844,3 +844,5 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b11484–b11487 | patches + upstream verification | **All ten patches apply unchanged**, verified in order against pristine b11487 with `git apply`. No patch-target file in the range; nothing under `src/`, `tools/server/`, `common/arg.cpp`, `common/log.h` or `include/` changed, so every drop-check of the b11480 row holds unchanged. | | b11487–b11491 | Four commits, 28 files, ~1104/70 lines (92 KiB, one step). **#30114 adds the LiquidAI d1-omni-600M decision model** (audio and image input): `COMMON_DECISION_TYPE_LFM2_D1_OMNI` (`"lfm2-d1-omni"`, laya-style with another prompt layout), a memory-less LFM2 graph for the non-causal case (`src/models/lfm2.cpp`, +295; `models.h`, `llama-model.cpp`), mtmd's conformer GLU sigmoid kept on CUDA and new `mtmd-audio` helpers, and in the server audio arrives through `images` / `input_audio`, with `images` now an alias of `files` on `/v1/systemone` (`server-decision.{h,cpp}`, +233; README). `LlamaModel.handleSystemOne` hands the request body through, so the alias and the audio input are available to Java callers as they are. #29507 removes duplicate SYCL block-size defines; #30126/#30121 Hexagon (`alloc_buffer_n`, Q6_K dequant). **No project source change.** | | b11487–b11491 | patches + upstream verification | **All ten patches apply unchanged**, verified in order against pristine b11491 with `git apply`. Patch-target files in the range: `src/llama-model.cpp` and `src/models/models.h` (`0012`, `0016`: the LFM2 additions are away from `load_tensors`' split block and from the Kolibri-1 registration and struct) and `tools/server/server-context.cpp` (`0002`, `0003`; a two-line change in the decision path). Drop-checks unchanged: still no `split_sum` guard, still no `kolibri` under `src/`. | +| b11491–b11495 | Four commits, 19 files, ~1461/43 lines (97 KiB, one step). **#29692 makes the reranker classifier activation a model property**: the new GGUF key `.classifier.activation` (`LLM_KV_CLASSIFIER_ACTIVATION`, `src/llama-arch.{h,cpp}`) sets `hparams.act_cls` (`"gelu"` → `GELU_ERF`, `"silu"`, `"tanh"`; anything else asserts at load), and `build_pooling` applies it to the RANK head instead of the hard-coded ModernBERT-gelu / otherwise-tanh choice; the default stays `tanh`, ModernBERT without the key keeps `gelu_erf`, so a GGUF converted before this (the CI jina reranker included) scores exactly as before, and `handleRerank` is unchanged. #27920 teaches the converter Qwen3.5 embedding models (Python plus a pooling tweak in `src/models`); #30087 CUDA GDN state columns per warp; #29245 SYCL grouped MoE XMX GEMM. **No project source change.** | +| b11491–b11495 | patches + upstream verification | **All ten patches apply unchanged**, verified in order against pristine b11495 with `git apply`. Patch-target files in the range: `src/llama-arch.{cpp,h}` and `src/llama-model.cpp` (`0016` adds the Kolibri-1 arch entries next to the new classifier key; `0012`'s split block is untouched). Drop-checks unchanged. | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index 22383897b..e0f1f6333 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -216,7 +216,7 @@ endif() FetchContent_Declare( llama.cpp GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git - GIT_TAG b11491 + GIT_TAG b11495 PATCH_COMMAND ${CMAKE_COMMAND} -DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches -DLLAMA_SRC= diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index b0047b1ea..8ef3a5055 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -9,13 +9,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11491"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11495"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b11491-"} — call + * plus the resolved upstream commit, e.g. {@code "b11495-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -23,14 +23,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11491"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11495"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b11491"; + public static final String LLAMA_CPP_VERSION = "b11495"; // Constants holder — not instantiable. private LlamaCppVersion() {} From 80517a1cbfa9886de87289606fed46d3a005c32a Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 8 Oct 2026 19:10:18 +0000 Subject: [PATCH 7/8] Upgrade llama.cpp from b11495 to b11505 All ten patches apply unchanged (verified in order against the pristine tag); see docs/history/llama-cpp-breaking-changes.md for the range. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2 --- CLAUDE.md | 8 ++++---- README.md | 2 +- docs/history/llama-cpp-breaking-changes.md | 2 ++ llama/CMakeLists.txt | 2 +- .../java/net/ladenthin/llama/value/LlamaCppVersion.java | 8 ++++---- 5 files changed, 12 insertions(+), 10 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 5a566cb11..6a43f6051 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b11495** +Current llama.cpp pinned version: **b11505** ## Natives jars: one directory per backend (`.github/natives.csv`) @@ -787,7 +787,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network for the asset build; the embed step is plain cmake -P -git clone --depth 1 --branch b11495 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b11505 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build ) mkdir -p webui-generated /tmp/ui-gen cmake -DUI_SOURCE_DIR=/tmp/lc/tools/ui -DUI_BINARY_DIR=/tmp/ui-gen \ @@ -827,7 +827,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b11495`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b11505`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -1966,7 +1966,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b11495`. +llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b11505`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.18.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index 8e1482971..8b6c3e090 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b11495](https://img.shields.io/badge/llama.cpp-%23b11495-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11495) +[![llama.cpp b11505](https://img.shields.io/badge/llama.cpp-%23b11505-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11505) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index 428d167ab..e70d83138 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -846,3 +846,5 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b11487–b11491 | patches + upstream verification | **All ten patches apply unchanged**, verified in order against pristine b11491 with `git apply`. Patch-target files in the range: `src/llama-model.cpp` and `src/models/models.h` (`0012`, `0016`: the LFM2 additions are away from `load_tensors`' split block and from the Kolibri-1 registration and struct) and `tools/server/server-context.cpp` (`0002`, `0003`; a two-line change in the decision path). Drop-checks unchanged: still no `split_sum` guard, still no `kolibri` under `src/`. | | b11491–b11495 | Four commits, 19 files, ~1461/43 lines (97 KiB, one step). **#29692 makes the reranker classifier activation a model property**: the new GGUF key `.classifier.activation` (`LLM_KV_CLASSIFIER_ACTIVATION`, `src/llama-arch.{h,cpp}`) sets `hparams.act_cls` (`"gelu"` → `GELU_ERF`, `"silu"`, `"tanh"`; anything else asserts at load), and `build_pooling` applies it to the RANK head instead of the hard-coded ModernBERT-gelu / otherwise-tanh choice; the default stays `tanh`, ModernBERT without the key keeps `gelu_erf`, so a GGUF converted before this (the CI jina reranker included) scores exactly as before, and `handleRerank` is unchanged. #27920 teaches the converter Qwen3.5 embedding models (Python plus a pooling tweak in `src/models`); #30087 CUDA GDN state columns per warp; #29245 SYCL grouped MoE XMX GEMM. **No project source change.** | | b11491–b11495 | patches + upstream verification | **All ten patches apply unchanged**, verified in order against pristine b11495 with `git apply`. Patch-target files in the range: `src/llama-arch.{cpp,h}` and `src/llama-model.cpp` (`0016` adds the Kolibri-1 arch entries next to the new classifier key; `0012`'s split block is untouched). Drop-checks unchanged. | +| b11495–b11505 | Ten commits, 19 files, ~821/286 lines (95 KiB, one step), backends and the vendored HTTP library. **cpp-httplib 0.60.1** (#30134): a `Range` suffix longer than the representation selects the whole representation (RFC 9110), the Windows certificate-chain check no longer rejects on `dwErrorStatus`, a `WebSocketClient` friend declaration -- compiled into `jllama` through `httplib.cpp`, none of it on the embedded server's path (no range requests, no TLS). **Vulkan:** `TOP_K` no longer hangs or returns wrong indices for +inf/NaN inputs and orders negative values correctly for k = 1 (#30107; a correctness fix for the Vulkan jars), sparse flash attention extended to coopmat2 (#30003). **CUDA:** norm-family kernels beyond the grid-dimension limits (#28175), arbitrary strides for f16/f32/bf16 unary ops (#29781) and `ROLL` (#29547). **SYCL:** FWHT (#29605), the fused delta-net alpha gate (#29687), model loading staged through a pinned ring buffer (#29608). Hexagon mmadd alignment (#30133). **No project source change.** | +| b11495–b11505 | patches + upstream verification | **All ten patches apply unchanged**, verified in order against pristine b11505 with `git apply`. No patch-target file in the range; nothing under `src/`, `common/`, `tools/server/` or `include/` changed, so every drop-check of the b11480 row holds unchanged. | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index e0f1f6333..9b2e06016 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -216,7 +216,7 @@ endif() FetchContent_Declare( llama.cpp GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git - GIT_TAG b11495 + GIT_TAG b11505 PATCH_COMMAND ${CMAKE_COMMAND} -DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches -DLLAMA_SRC= diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index 8ef3a5055..ba01e2b35 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -9,13 +9,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11495"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11505"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b11495-"} — call + * plus the resolved upstream commit, e.g. {@code "b11505-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -23,14 +23,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11495"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11505"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b11495"; + public static final String LLAMA_CPP_VERSION = "b11505"; // Constants holder — not instantiable. private LlamaCppVersion() {} From 51be68665056d8ce4c192742b177966f83c8efa3 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 8 Oct 2026 19:10:18 +0000 Subject: [PATCH 8/8] Upgrade llama.cpp from b11505 to b11512 All ten patches apply unchanged (verified in order against the pristine tag); see docs/history/llama-cpp-breaking-changes.md for the range. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2 --- CHANGELOG.md | 8 ++++++++ CLAUDE.md | 8 ++++---- README.md | 2 +- docs/history/llama-cpp-breaking-changes.md | 2 ++ llama/CMakeLists.txt | 2 +- .../java/net/ladenthin/llama/value/LlamaCppVersion.java | 8 ++++---- 6 files changed, 20 insertions(+), 10 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6cc3a6813..990055e6b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -79,6 +79,14 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by GitHub retires the `macos-14` image by 2026-11-02 and fails every `macos-14` job during its October brownouts. The label is pinned rather than `macos-latest`, and it is not `macos-15`, which the shipped Metal build and the no-Metal build already run on. +- **Upgraded the pinned llama.cpp from b11476 to b11512**, in 7 reviewed steps, each ending at a tag. + No carried patch needed a refresh, and every drop-check still finds its defect. The chat parser now + receives the generated tokens next to the text (#29876, `common_chat_input`) -- a change in the C++ + tests here, not on the wire: request fields, their bounds and the response keys are identical across + the range. Two things a user can notice: the embedded server speaks cpp-httplib 0.60.1 (b11505), and + a slot state file written from b11512 on carries the context checkpoints (#26004) in an appendix an + older file simply lacks, so files from earlier releases still restore. The new `--moe-cache-mib` is + under *Added*. Per-step record: `docs/history/llama-cpp-breaking-changes.md`. - **Upgraded the pinned llama.cpp from b11320 to b11476**, in 25 reviewed steps, each ending at a tag. Every carried patch that broke was traced to the one upstream commit that broke it, and the step containing that commit ends at the first tag after it: `0007` at #29818 (b11361) and #29895 (b11401), `0014` at #29895, `0008` diff --git a/CLAUDE.md b/CLAUDE.md index 6a43f6051..98436f021 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b11505** +Current llama.cpp pinned version: **b11512** ## Natives jars: one directory per backend (`.github/natives.csv`) @@ -787,7 +787,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network for the asset build; the embed step is plain cmake -P -git clone --depth 1 --branch b11505 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b11512 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build ) mkdir -p webui-generated /tmp/ui-gen cmake -DUI_SOURCE_DIR=/tmp/lc/tools/ui -DUI_BINARY_DIR=/tmp/ui-gen \ @@ -827,7 +827,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b11505`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b11512`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -1966,7 +1966,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b11505`. +llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b11512`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.18.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index 8b6c3e090..1fe4e1cd9 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b11505](https://img.shields.io/badge/llama.cpp-%23b11505-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11505) +[![llama.cpp b11512](https://img.shields.io/badge/llama.cpp-%23b11512-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11512) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index e70d83138..8f9646489 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -848,3 +848,5 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b11491–b11495 | patches + upstream verification | **All ten patches apply unchanged**, verified in order against pristine b11495 with `git apply`. Patch-target files in the range: `src/llama-arch.{cpp,h}` and `src/llama-model.cpp` (`0016` adds the Kolibri-1 arch entries next to the new classifier key; `0012`'s split block is untouched). Drop-checks unchanged. | | b11495–b11505 | Ten commits, 19 files, ~821/286 lines (95 KiB, one step), backends and the vendored HTTP library. **cpp-httplib 0.60.1** (#30134): a `Range` suffix longer than the representation selects the whole representation (RFC 9110), the Windows certificate-chain check no longer rejects on `dwErrorStatus`, a `WebSocketClient` friend declaration -- compiled into `jllama` through `httplib.cpp`, none of it on the embedded server's path (no range requests, no TLS). **Vulkan:** `TOP_K` no longer hangs or returns wrong indices for +inf/NaN inputs and orders negative values correctly for k = 1 (#30107; a correctness fix for the Vulkan jars), sparse flash attention extended to coopmat2 (#30003). **CUDA:** norm-family kernels beyond the grid-dimension limits (#28175), arbitrary strides for f16/f32/bf16 unary ops (#29781) and `ROLL` (#29547). **SYCL:** FWHT (#29605), the fused delta-net alpha gate (#29687), model loading staged through a pinned ring buffer (#29608). Hexagon mmadd alignment (#30133). **No project source change.** | | b11495–b11505 | patches + upstream verification | **All ten patches apply unchanged**, verified in order against pristine b11505 with `git apply`. No patch-target file in the range; nothing under `src/`, `common/`, `tools/server/` or `include/` changed, so every drop-check of the b11480 row holds unchanged. | +| b11505–b11512 | Seven commits, 25 files, ~716/202 lines (79 KiB, the final step). **#26004 keeps the context checkpoints across a slot save and restore**: the slot file gains an appendix after the llama state payload (magic `SCKP`, version 1, a count, then per checkpoint `n_tokens`/`pos_min`/`pos_max` and the target, draft and speculative blobs), counted in the `n_written` / `n_read` of the save and restore results; on restore it is read bounds-checked -- a missing, truncated or foreign appendix is ignored with a warning, never fatal, so a slot file written before this restores as before -- and one draft checkpoint is test-loaded, the draft data dropped when it does not fit the current draft context, where `common_prompt_checkpoint::load_dft()` used to `GGML_ABORT` on a size mismatch (`load_tgt`/`load_dft` return `bool` now, `common/common.h`). `LlamaModel.handleSlotAction(SAVE/RESTORE)` carries the same payload; the response keys are unchanged. **#30112 splits the MoE expert cache across several GPUs** like the layers (`--tensor-split`; the `--moe-cache-mib` help text, `common_params`, `llama-moe-cache.cpp`) -- `setMoeCacheMib`'s Javadoc already says so. **CUDA:** MMQ out-of-bounds reads (#29953), a looped PAD kernel beyond 65535 rows or slices (#30147), and the CCCL version guard in `argsort.cu` compares the packed `CCCL_VERSION` (#29453: the per-component compare turned `STRIDED_ITERATOR_AVAILABLE` off on a major rollover -- with the pinned CCCL v3.4.3 it is true before and after, so no change for the CUDA jars until CCCL 4). #30111 reads DFlash's tied output weights from the GGUF metadata; #29668 WebUI settings on the first visit in router mode (auto-followed). **No project source change.** | +| b11505–b11512 | patches + upstream verification | **All ten patches apply unchanged**, verified in order against pristine b11512 with `git apply` and by a fresh configure through the real `FetchContent` path. Patch-target files in the range: `common/arg.cpp` (`0001`, `0015`: a help-text change only) and `tools/server/server-context.cpp` (`0002`, `0003`: the checkpoint appendix helpers are private members added away from `load_model` and the similarity getter/setter). Drop-checks against pristine b11512 -- the final state of the bump: `common_params_parse` still carries the `#ifdef _WIN32` `argv = utf8.ptrs.data()` override and `common_params_parse_main` is absent (`0001`); the load-progress assignment is unconditional (`0002`); no `slot_prompt_similarity` getter (`0003`); no embedded-mode, attach or worker-command counterpart (`0006`-`0008`); no `split_sum == 0` guard (`0012`); `common/log.h` names only `common_log_default_callback`, no sink (`0014`); no `stop_server` and the connect failure still aborts in `ggml-rpc.cpp` (`0015`); no `kolibri` under `src/` (`0016`). No `.github/` change upstream in the whole range. | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index 9b2e06016..7371cc830 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -216,7 +216,7 @@ endif() FetchContent_Declare( llama.cpp GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git - GIT_TAG b11505 + GIT_TAG b11512 PATCH_COMMAND ${CMAKE_COMMAND} -DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches -DLLAMA_SRC= diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index ba01e2b35..54f6aea8e 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -9,13 +9,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11505"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11512"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b11505-"} — call + * plus the resolved upstream commit, e.g. {@code "b11512-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -23,14 +23,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11505"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11512"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b11505"; + public static final String LLAMA_CPP_VERSION = "b11512"; // Constants holder — not instantiable. private LlamaCppVersion() {}