bernardladenthin · Oct 14, 2026 · Oct 14, 2026
@@ -11,7 +11,7 @@
 **Build:**  
 ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational)  
 ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey)  
-[![llama.cpp b11574](https://img.shields.io/badge/llama.cpp-%23b11574-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11574)  
+[![llama.cpp b11612](https://img.shields.io/badge/llama.cpp-%23b11612-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11612)  
 [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/)  
 ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162)  
 [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev)  

@@ -842,6 +842,57 @@ whose "17 issues found" status once posted on PRs #248/#298; it posts nothing on
 never set up deliberately. Whether the project wants one, with which policy, is the owner's call;
 nothing here blocks on it.
 
+### Drop patch `0012` at the first bump past b11612 (upstream merged the fix as #29677)
+
+**The drop-check of `0012` fires at the *next* bump, not at the b11612 one.** On 2026-10-14 upstream
+merged [ggml-org/llama.cpp#29677](https://github.com/ggml-org/llama.cpp/pull/29677), commit
+[`df18bee11`](https://github.com/ggml-org/llama.cpp/commit/df18bee11e96557b07aa7b12db0976f3c7ac9165)
+("model: split layers evenly when no device reports free memory"), one line in `load_tensors()`:
+
+```cpp
+splits[i] = split_sum != 0.0f ? splits[i] / split_sum : float(i + 1) / n_devices();
+```
+
+That is the even-split fallback `patches/0012` carries in `llama_model_splits_normalize()`, 15 commits
+**after** b11612 (`10da8c463`) on master -- at b11612 `src/llama-model.cpp:1648` is still the bare
+`splits[i] /= split_sum` (measured on the pristine tag), so the patch is still carried at this pin and
+applies unchanged. The PR touches `src/llama-model.cpp` only: no warning, no diagnostic on the device
+lookup, **no test**.
+
+**The rule for the bump that crosses it:**
+
+1. `git grep -n split_sum src/llama-model.cpp` at the new tag. The guarded line is there → **drop
+   `0012`, do not refresh it.** The applier will not say so: `0012` applies *next to* upstream's line
+   (its hunks lift the two blocks out of `load_tensors()` and touch other lines), so it keeps applying
+   cleanly -- the `0010` case, "still applies, no longer needed". Delete the patch file; its `CLAUDE.md`
+   row moves to `docs/history/dropped-llama-patches.md` with the note below.
+2. What goes with it, deliberately: the `LLAMA_LOG_WARN` naming the cause, the bounds check in
+   `llama_model_splits_select_device()` whose message names layer, index and split points (upstream keeps
+   `devices.at()`, so a malformed split still reads `"vector"`), and the never-compiled upstream
+   `tests/test-model-split.cpp`. None of that is worth a patch on its own. One thing upstream's line does
+   **not** cover, for the record: a NaN weight (`-ts nan,1` -- `std::stof` accepts it, `NaN != 0.0f` is
+   true) still produces NaN split points and the bare `"vector"`; `0012`'s message named it. If that ever
+   matters it is a separate upstream issue, not a reason to keep the patch.
+3. **The guard `src/test/cpp/test_model_split.cpp` cannot stay as it is.** It links
+   `llama_model_splits_normalize` and `llama_model_splits_select_device`, which exist only through the
+   patch -- without it `jllama_test` fails to link on every platform. That was the design (a dropped
+   patch reds `C++ Tests`), and at the drop it works against us; and upstream shipped no test the guard
+   could be re-pointed at. So the guard is replaced by one over upstream's contract, like `0010` / `0011`:
+   - **Preferred: an end-to-end test through the real load path**, the only public route to the fixed
+     line. The backend-independent trigger the `0012` row documents is `--tensor-split 1,-1`, which needs
+     **two** devices: two loopback RPC servers serving the CPU (`test_rpc.cpp`'s `loopback_server`)
+     register two GPU-type devices; load a tiny GGUF written with the `gguf` API (as `test_kolibri1.cpp`
+     does) with `tensor_split = {1, -1}` and every layer offloaded. At an unfixed tag the load fails
+     (`std::out_of_range` from `devices.at()`, the bare `"vector"` in the log); at a fixed one it loads
+     with the layers split evenly. That pins upstream's line on every platform in `C++ Tests` without a
+     patch. **Verify first** that two `ggml_backend_rpc_start_server` instances coexist in one process
+     (`0015`'s stop mechanism is global state); if they cannot, this option is out.
+   - **Otherwise the guard goes with the patch**, and the drop note says why no unit-level guard is
+     possible (the line sits inside a member function with no seam). Do **not** keep a refactor-only
+     residue of `0012` just so the test keeps linking: that is a patch to refresh on every bump for a line
+     upstream owns, the exact thing the drop rule exists to avoid.
+4. `patches/0012`'s header and `test_model_split.cpp`'s header carry the same pointer to this section.
+
 ### Upstream submissions -- not before the release (policy)
 
 **Nothing is filed upstream before the release is out**, neither issues nor PRs; the drafts are not
@@ -885,15 +936,16 @@ treat an unknown flag as pass-through). So the measurement stands as recorded an
 to add this case to one of those issues, or wait. Re-check on the next sccache release before
 spending a CI run on it.
 
-There are **twelve** patches today (`0001`-`0003`, `0006`-`0008`, `0012`, `0014`-`0018`). Each
+There are **thirteen** patches today (`0001`-`0003`, `0006`-`0008`, `0012`, `0014`-`0019`). Each
 accepted upstream change deletes a patch from the bump checklist once the pin is bumped past it.
 
 - **Submittable as they are** (self-contained, no jllama specifics): `0001` (Windows arg-parse embed
   guard, already reported as ggml-org/llama.cpp#26416 with the reproducer in
   `docs/upstream-investigation-win32-argv-substitution.md`; the maintainers have not picked a
   direction), `0002` (preserve the caller's load-progress callback), `0006` (embeddable
-  `llama_server`), `0007` (`llama_server_attach`), `0008` (`LLAMA_SERVER_WORKER_CMD`), `0012` (zero
-  split-sum guard, ships `tests/test-model-split.cpp`), `0014` (`common_log_set_callback`), `0015`
+  `llama_server`), `0007` (`llama_server_attach`), `0008` (`LLAMA_SERVER_WORKER_CMD`), ~~`0012`~~ (zero
+  split-sum guard -- **fixed upstream independently on 2026-10-14 by #29677**, nothing left to submit;
+  dropped at the first bump past b11612, see the section right above), `0014` (`common_log_set_callback`), `0015`
   (non-aborting RPC registration, stoppable server, transport fixes), `0017` (x86 prefetch helper),
   **`0018`** (the AMX buffer released with `free()` instead of `ggml_aligned_free` -- one line, with
   the 6/6-against-0/6 measurement in its header and a configure-time guard that is red without it;

@@ -886,3 +886,5 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r
 | b11570–b11573 | patches + upstream verification | All twelve patches apply (`git apply` in order on a pristine b11573 worktree): `0016` (kolibri1, `src/llama-arch.{h,cpp}`, `src/llama-model.cpp`, `src/models/models.h`) applies next to the new architecture's entries at their own positions, `0012` (refreshed in the previous step) next to the model additions in `src/llama-model.cpp`. Drop-checks against pristine b11573/b11574: `src/llama-model.cpp` changed in the range and still sums and divides by `split_sum` unguarded (`:1642-1648`, `0012`); no `kolibri` under `src/` (`0016`); `common/log.h` still names only `common_log_default_callback` (`0014`); no `stop_server`, the connect failure still aborts at `ggml-rpc.cpp:645` (`0015`); the four `_mm_prefetch` calls at `quants.c:782-801` still uncast (`0017`); `amx.cpp:46` still `free(buffer->context)` (`0018`); `arg.cpp` still carries the `GetCommandLineW` override and no `common_params_parse_main` (`0001`); `server-context.cpp:1212` still assigns the load-progress callback unconditionally (`0002`); no `LLAMA_SERVER_WORKER_CMD` (`0008`), no `get_slot_prompt_similarity` (`0003`) upstream. |
 | b11573–b11574 | 1 commit, 10 files, ~822/128 lines (58 KiB, the target of the series; no `tools/ui` change). **`ggml/src/ggml-hexagon/`** (#30341 "hexagon: optimize CPY/CONCAT — round 3, short-rows edition"): the Qualcomm Hexagon HTP backend's copy and concat ops gain a gather-rows path for short rows (`htp/gather-rows.{c,h}`, new; `cpy-ops` / `concat-ops` dispatch to it), the backend's own `htp/CMakeLists.txt` lists the file, and `scripts/snapdragon/build.py` grows options -- a backend this project does not build (no `hexagon` row in `natives.csv`; the Android jars carry the CPU modules and OpenCL only), so nothing in the range is compiled here. Root `CMakeLists.txt`, `tools/server/`, `common/`, `src/` and every patch-target file are outside the range. **No project source change.** |
 | b11573–b11574 | patches + upstream verification | **All twelve patches apply unchanged at the target** (`check-patches.py`: 12 patches, 99 hunks, 0 problems; the real applier `apply-llama-patches.cmake` on a fresh `FetchContent` checkout of b11574 -- `rm -rf build`, so the fetch, the twelve patches and the stamp (`head 8bb3cf0d2`, 58 modified files in the source tree) went through the production path -- then the static test build, `ctest` 607/607, and the model-free native Java tests over it: `NativeLibraryLoadSmokeTest` (the `LlamaCppVersion` pin against the linked build's `b11574-<commit>`), `LlamaLoggerTest`, `RpcServerTest`, 19/19). **The modular build** (`-DJLLAMA_BACKEND_DL=ON`, the shape of every Linux / Windows / Android library jar) **is where #30014's hidden visibility (b11570 row) could have bitten, and did not**: `libggml-base.so` exports 689 functions and `libggml.so` 16 now, every one `GGML_API` / `GGML_BACKEND_API`, among them the five the load path needs (`ggml_backend_load_all_from_path`, `ggml_backend_load`, `ggml_backend_reg_get_proc_address`, `ggml_backend_dev_count`, `ggml_log_set`); the CPU modules and `ggml-rpc` are `MODULE` targets outside the preset and export their entry points as before (390 in `libggml-cpu-haswell.so`, 77 in `libggml-rpc.so`); `JNI_OnLoad` registered RPC and CPU from the extraction directory (`2 backend(s) registered ... [RPC, CPU]`, `cascadelake` chosen on the sandbox's Xeon), and `RpcServerTest`, `LlamaLoggerTest`, `BackendLoadTest` and `NativeLibraryLoadSmokeTest` ran 27/27 over the modular library -- the RPC server resolved through `ggml_backend_reg_get_proc_address` included. Drop-checks against pristine b11574 are those of the b11573 row, the range touching no patch-target file: `split_sum` unguarded (`0012`), no `kolibri` (`0016`), no `common_log` hook (`0014`), no `stop_server` (`0015`), uncast `_mm_prefetch` (`0017`), `free(buffer->context)` (`0018`), the `GetCommandLineW` override and no `common_params_parse_main` (`0001`), the unconditional load-progress assignment (`0002`), no `LLAMA_SERVER_WORKER_CMD` (`0008`), no `get_slot_prompt_similarity` (`0003`). `check-natives.py`: 23 natives jars, 0 disagreements. Not run locally: the model-backed Java suite (no GGUF in the sandbox) -- CI's test jobs cover it. |
+| b11574–b11612 | 38 commits, 287 files, ~27,300/17,750 lines, 6.4 MB -- bumped in one step rather than in the runbook's < 100 KiB chunks because the bulk is data: `docs/ops/zDNN.csv` (31,433 lines), `src/unicode-data.cpp` (+3,985, #30390 opt-in NFC normalization for Unigram tokenizers), the consolidated context-state tests (#30023: `tests/test-save-load-state.cpp`, `test-recurrent-state-rollback.cpp` and `test-state-restore-fragmented.cpp` become `tests/test-llama-context.cpp` + `tests/llama-context/`), kleidiai (#29219, SME2 Q4_K/Q6_K, `KLEIDIAI_COMMIT_TAG` v1.24.0 → v1.31.0 -- ARM only, not built here) and the Vulkan/CUDA/Hexagon/zDNN backends; `tools/ui` unchanged. **Priority list:** `common/chat.h` (#30389 "chat: replace output string with parts"): `common_chat_params::prompt` and `common_chat_session::prompt()` are a `common_chat_prompt` -- a vector of `{text, is_input}` parts, `str()` renders it, `append()` / `replace_last()` skip user-input parts -- and `common_chat_template_direct_apply()` returns one; `server-common.cpp` reads `out_session.prompt().str()`. `jllama.cpp` threads the session through but never reads the prompt, so **no project source change**. `common/log.{h,cpp}` (#30023, of all things): `common_log_set_stdo()` + a `stdo` member, console output gated on it -- context of `0014`, see the patches row. `include/llama.h`: a comment on `embeddings` only; behind it #30445 ("graph: skip the LM head for pooled embeddings"): a pooled context (`pooling_type != NONE`) no longer computes logits when `embeddings` is on, `llama_set_embeddings()` toggles the graph shape (`sched_need_reserve`), and `has_logits = !cparams.embeddings || graph_has_logits` -- the embedding path only, which `LlamaEmbeddingsTest` and the langchain4j integration job exercise in CI. `tools/server/`: #30243 (`server_decision_escape_special_tokens`, `<|name|>` written with U+00A6 instead of `std::regex`) -- the decision path, no schema or response-key change (`server-schema.cpp` field set and `server-task.cpp` emit keys identical at both tags, the two mechanical checks). `src/llama-model.cpp` (#30405 "do not map foreign ranges into a backend buffer": `get_mapping_ranges()` replaces `get_mapping_range()`, `ctx_buf_maps` becomes `ctx_bufs`; #30445's `build_logits()`; `llama_model_set_attn_rot`): outside `0012`'s and `0016`'s hunks, both apply unchanged. `ggml/src/ggml-backend-reg.cpp` (#28367): `ggml_backend_load_best()` drops the pre-checks on each search path and reports an enumeration error once after the loop -- `JNI_OnLoad` calls `ggml_backend_load_all_from_path()`, unaffected. `ggml/src/ggml-cpu/arch/x86/quants.c` (+64, #30447 AVX2 Q2_0 dot product): away from `0017`'s four `_mm_prefetch` lines. Toolchain pins in upstream's `release.yml` unchanged: OpenVINO `2026.4.1` / `2026.4.1.22982.07f9c262b05`, `GGML_CUDA_CCCL_VERSION=v3.4.3`, ROCm `10.0.0`; BoringSSL `0.20260929.0`; `vendor/` untouched (cpp-httplib 0.61.0, #30494, is after b11612). Root `CMakeLists.txt` unchanged, `GGML_LLAMAFILE_DEFAULT` / `GGML_CUDA_GRAPHS_DEFAULT` still the only ggml defaults it sets. **No project source change.** |
+| b11574–b11612 | patches + upstream verification | **Three of the thirteen patches needed a refresh, none was dropped.** `0001`: `tests/test-state-restore-fragmented.cpp` is gone (#30023 consolidated the context-state tests into `tests/test-llama-context.cpp`), so its hunk was dropped (36 → 35 `diff --git` headers); the consolidated test has the `--models` / `filtered_argv` `main()` and by the patch's own rule keeps `common_params_parse`. `0014`: #30023 also added `common_log_set_stdo()` and a `stdo` member exactly where the sink's members and `print_entry()`'s console call sit -- three hunks regenerated, the sink keeps precedence (`if (callback) ... else if (stdo) e.print()`), pinned by a seventh case in `test_common_log_callback.cpp`. `0006` failed only in the cascade behind `0001` (its context is the post-`0001` tree) and applies unchanged once `0001` does. The other ten apply unchanged. `check-patches.py`: 13 patches, 101 hunks, 0 problems; the real applier on a fresh `FetchContent` checkout of b11612 (stamp head `10da8c463`, 57 modified files plus the stamp), then the static test build, `ctest` **608/608**, and the model-free native Java tests over the library (`NativeLibraryLoadSmokeTest` with the `LlamaCppVersion` pin against the linked `b11612-<commit>`, `LlamaLoggerTest`, `RpcServerTest`, `BackendLoadTest`: 27/27). **Drop-checks against pristine b11612, and the one that will fire next:** `src/llama-model.cpp:1648` is still the bare `splits[i] /= split_sum` (`0012`) -- **but upstream merged the fix the same day, 15 commits past b11612**: [#29677](https://github.com/ggml-org/llama.cpp/pull/29677) / `df18bee11` ("model: split layers evenly when no device reports free memory"), the identical even-split fallback and no test. The first bump past it drops `0012`; the plan for its guard `test_model_split.cpp`, which links the two functions only the patch provides, is in `TODO.md` ("Drop patch `0012` at the first bump past b11612"). The rest unchanged: no `kolibri` under `src/` (`0016`); `common/log.h` has a `set_stdo` but no hook, only `common_log_default_callback` (`0014`); no `stop_server`, the connect failure still aborts at `ggml-rpc.cpp:645` (`0015`); the four `_mm_prefetch` calls at `quants.c:846-865` still uncast (`0017`); `amx.cpp:46` still `free(buffer->context)` (`0018`); the `__ARM_NEON` branch of `simd-mappings.h` still selects the software conversion (`0019`); `arg.cpp:1288` still carries the `GetCommandLineW` override and no `common_params_parse_main` (`0001`); `server-context.cpp:1212` still assigns the load-progress callback unconditionally (`0002`); no `LLAMA_SERVER_WORKER_CMD` (`0008`), no `get_slot_prompt_similarity` (`0003`). `check-natives.py`: 23 natives jars, 0 disagreements; spotless clean. Not run locally: the model-backed Java suite (no GGUF in the sandbox) and the modular build -- CI's test and smoke jobs cover both. |
@@ -337,7 +337,7 @@ endif()
 FetchContent_Declare(
 	llama.cpp
 	GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git
-	GIT_TAG        b11574
+	GIT_TAG        b11612
 	PATCH_COMMAND  ${CMAKE_COMMAND}
 		-DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches
 		-DLLAMA_SRC=<SOURCE_DIR>

@@ -300,19 +300,6 @@ index 46ded1398..d5f1cabe7 100644
          return 1;
      }
 
-diff --git a/tests/test-state-restore-fragmented.cpp b/tests/test-state-restore-fragmented.cpp
-index d5548afba..95ba2b67b 100644
---- a/tests/test-state-restore-fragmented.cpp
-+++ b/tests/test-state-restore-fragmented.cpp
-@@ -24,7 +24,7 @@ int main(int argc, char ** argv) {
-
-     common_init();
-
--    if (!common_params_parse(argc, argv, params, LLAMA_EXAMPLE_COMMON)) {
-+    if (!common_params_parse_main(argc, argv, params, LLAMA_EXAMPLE_COMMON)) {
-         return 1;
-     }
-
 diff --git a/tests/test-thread-safety.cpp b/tests/test-thread-safety.cpp
 index d0b5946e2..5b665b426 100644
 --- a/tests/test-thread-safety.cpp

@@ -1,3 +1,20 @@
+model: fall back to an even split when no device reports free memory, and name the device index
+on a bad lookup (the two load_tensors() blocks lifted into free functions so they can be tested).
+
+STATUS 2026-10-14: FIXED UPSTREAM -- DROP THIS PATCH AT THE FIRST llama.cpp BUMP PAST b11612.
+ggml-org/llama.cpp#29677, commit df18bee11 ("model: split layers evenly when no device reports
+free memory"), https://github.com/ggml-org/llama.cpp/commit/df18bee11e96557b07aa7b12db0976f3c7ac9165,
+puts the same even-split fallback into load_tensors():
+
+    splits[i] = split_sum != 0.0f ? splits[i] / split_sum : float(i + 1) / n_devices();
+
+It is NOT in b11612 (10da8c463, 15 commits earlier on master), so the patch is still needed at that
+pin and applies unchanged. Drop-check: `git grep -n split_sum src/llama-model.cpp` at the new tag --
+the guarded line means DROP, not refresh (the patch keeps applying next to it, so the applier will
+not notice). Upstream ships no test. The runnable guard src/test/cpp/test_model_split.cpp links the
+two functions this patch adds and cannot outlive it as it is; what replaces it is decided in
+TODO.md, "Drop patch 0012 at the first bump past b11612".
+
 diff --git a/src/llama-model.cpp b/src/llama-model.cpp
 index 6872589fa..a56e4bb04 100644
 --- a/src/llama-model.cpp