Upgrade llama.cpp from b11512 to b11529; nvcc through sccache opt-in; docs cleanup

.github/build.sh

@@ -105,18 +105,29 @@ if [ "${USE_CACHE:-true}" = "true" ] && command -v sccache >/dev/null 2>&1 \
    && [ -n "${SCCACHE_WEBDAV_TOKEN:-}${SCCACHE_GHA_ENABLED:-}" ] \
    && sccache_can_wrap_compiler; then
   LAUNCH="-DCMAKE_C_COMPILER_LAUNCHER=sccache -DCMAKE_CXX_COMPILER_LAUNCHER=sccache"
-  # CUDA builds: also wrap nvcc so the per-arch .cu device passes are cached too — not just
-  # the gcc host TUs. Those per-architecture device-pass objects are the dominant cost of the
-  # full-arch CUDA job, and sccache does support nvcc as a compiler. Scoped to CUDA builds
-  # (GGML_CUDA in the cmake args): CMAKE_CUDA_COMPILER_LAUNCHER is inert when CUDA is not an
-  # enabled language, but keeping it scoped leaves the CPU/Android jobs' configure output clean.
-  # If sccache cannot wrap nvcc it runs it directly (uncached); and the mid-build retry below
-  # also catches an sccache "Compiler not supported" failure and rebuilds without the launcher,
-  # so an nvcc-hostile sccache can never red the build.
+  # CUDA builds: nvcc through sccache is OPT-IN (SCCACHE_WRAP_NVCC=true), off by default.
+  # Wrapping nvcc cached the per-arch .cu device passes too, which are the dominant cost of the
+  # full-arch CUDA job (measured: ~51 min cold -> ~15 min warm with CUDA 13.2). Since the CUDA
+  # 13.4 toolkit (2026-09-27), sccache 0.18.0 fails on the very first .cu TUs, ten seconds into
+  # the build, in every job log examined (four of four: runs 37643964068, 37680715063,
+  # 37831180985, 37925157175): `Missing "cubin" file output` / `Compiler killed by signal 126`
+  # -- sccache's nvcc command-line parsing, not a compile error. The retry below then rebuilt
+  # EVERYTHING without a launcher, gcc TUs included (47-80 min depending on the runner). Not
+  # wrapping nvcc keeps the ~280 gcc objects cached and runs nvcc directly from the start, with
+  # no wasted first attempt. Re-try the opt-in on a PR run after bumping SCCACHE_DL_VERSION (or
+  # a CUDA bump); flip the default back once a warm run shows CUDA hits in the stats table.
+  # Scoped to CUDA builds (GGML_CUDA in the cmake args): CMAKE_CUDA_COMPILER_LAUNCHER is inert
+  # when CUDA is not an enabled language, but keeping it scoped leaves the CPU/Android jobs'
+  # configure output clean. ggml does not wrap nvcc on its own here: its GGML_CCACHE block only
+  # fires when neither CMAKE_C_COMPILER_LAUNCHER nor CMAKE_CXX_COMPILER_LAUNCHER is set.
   case " $* " in
     *" -DGGML_CUDA=1 "* | *" -DGGML_CUDA=ON "* | *" -DGGML_CUDA=on "*)
-      LAUNCH="$LAUNCH -DCMAKE_CUDA_COMPILER_LAUNCHER=sccache"
-      echo "build.sh: sccache will also wrap nvcc (CUDA build detected)" ;;
+      if [ "${SCCACHE_WRAP_NVCC:-false}" = "true" ]; then
+        LAUNCH="$LAUNCH -DCMAKE_CUDA_COMPILER_LAUNCHER=sccache"
+        echo "build.sh: sccache will also wrap nvcc (CUDA build detected, SCCACHE_WRAP_NVCC=true)"
+      else
+        echo "build.sh: nvcc runs uncached (CUDA build detected, SCCACHE_WRAP_NVCC is not 'true'); sccache wraps the C/C++ TUs only"
+      fi ;;
   esac
   echo "build.sh: sccache ON (endpoint=${SCCACHE_WEBDAV_ENDPOINT:-default}), building with -j${JOBS}"
 else

CHANGELOG.md

@@ -48,6 +48,24 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by
   core's; `check-natives.py` fails when they differ.
 
 ### Changed
+- **`NativeServer` binds port 9931 when no `--port` is given** (llama.cpp b11521,
+  [#30159](https://github.com/ggml-org/llama.cpp/pull/30159) moved the server default off 8080), in classic and
+  attach mode alike; `NativeServer.getPort()` reports 9931 accordingly. `OpenAiCompatServer` keeps its own default
+  of 8080 (`OpenAiServerConfig.DEFAULT_PORT`, a public constant of this project with no upstream coupling). Pass
+  `--port` explicitly where a fixed port matters; every README example already does.
+- **CI: the Linux CUDA build no longer routes nvcc through sccache by default** (`SCCACHE_WRAP_NVCC=true`
+  in `build.sh` opts back in). With CUDA 13.4 and sccache 0.18.0 the nvcc launcher failed on the first `.cu`
+  files in two confirmed runs (`Missing "cubin" file output`, `Compiler killed by signal 126`), and the
+  uncached retry that followed made those jobs slower than a cold build; the C/C++ objects stay cached.
+- **Upgraded the pinned llama.cpp from b11512 to b11529**, in 5 reviewed steps, each ending at a tag. No
+  carried patch needed a refresh, and every drop-check still finds its defect. In the range: the server's
+  default port moves from 8080 to 9931 (#30159, the `NativeServer` entry above); CUDA top-k selection is
+  reworked (#28713) and the MMQ config helpers take the `src1` precision (#30168); SYCL gains a Q5_K
+  reorder-layout MMVQ and a fused GLU (#29375); Metal gets flash-attention kernels for the 128/96 head-size
+  pair (#30209); Vulkan's `rms_norm` no longer overflows its workgroup count (#30145); `ggml_acc` no longer
+  writes out of bounds for a large offset (#30135); the meta backend handles host views under
+  `--split-mode tensor` (#30217); the WebUI gains a models manager (#29583, auto-followed by `build-webui`).
+  Per-step record: `docs/history/llama-cpp-breaking-changes.md`.
 - **Windows natives: no Visual C++ redistributable, and `ucrtbase.dll` stays serviceable — but
   Windows 10 or newer is now required.** The Windows CPU jars (`cpu-windows-*`,
   `msvc-windows-*`) and the CUDA, Vulkan, OpenCL and ROCm jars use the **hybrid CRT**: the C++

README.md

@@ -11,7 +11,7 @@
 **Build:**  
 ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational)  
 ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey)  
-[![llama.cpp b11512](https://img.shields.io/badge/llama.cpp-%23b11512-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11512)  
+[![llama.cpp b11529](https://img.shields.io/badge/llama.cpp-%23b11529-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11529)  
 [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/)  
 ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162)  
 [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev)  

TODO.md

@@ -166,14 +166,22 @@ binaries; 2026-10-08) established:
 7. **Unsigned binaries.** Upstream ships its DLLs unsigned, and so do we; on a WDAC/AppLocker-managed
    client, loading unsigned DLLs from `%TEMP%` is blocked. Out of scope here; worth a README note.
 
-### CUDA job: nvcc through sccache failed once and fell back
-
-- Run 37680715063 (b11476 bump): the Linux CUDA build logged `Missing cubin file output` and
-  `sccache: Compiler killed by signal 126` on `.cu` TUs, and `build.sh`'s retry rebuilt without the
-  launcher (green, ~50 min instead of ~15). Not the CUDA 13.3 `-virtual` failure already handled
-  (that one was `fatbinary ... acc.compute_75.ptx`). Check whether sccache 0.18.0 and CUDA 13.4 disagree
-  on the device-compile pipeline, or whether it was a transient cache-storage error; a second
-  occurrence makes the CUDA cache effectively off.
+### CUDA job: nvcc through sccache (`SCCACHE_WRAP_NVCC`, off since the second failure)
+
+- Run 37680715063 (b11476 bump) and run 37925157175 (#492, 2026-10-09): the Linux CUDA build logged
+  `sccache: caused by: Missing "cubin" file output` and `sccache: Compiler killed by signal 126` on
+  the first three `.cu` TUs, one minute into the build, and `build.sh`'s retry rebuilt everything
+  without any launcher (green; 86 min in total the second time, longer than a cold build). Not the
+  CUDA 13.3 `-virtual` `.ptx` failure, and not a storage error (the gcc probe passed and the gcc TUs
+  were being served): sccache 0.18.0 and CUDA 13.4's nvcc disagree on the device-compile command
+  line, in every job log examined since the 13.4 bump of 2026-09-27 (also runs 37643964068 and
+  37831180985; the retry alone takes 47-80 min depending on the runner, and no job came near the warm
+  ~15 min). Since the second confirmed occurrence `build.sh`
+  wraps nvcc only with `SCCACHE_WRAP_NVCC=true` (CLAUDE.md "Fast local CUDA builds"), so the gcc TUs
+  stay cached and nvcc runs directly from the start. **Open:** re-test the opt-in with a newer
+  sccache (bump `SCCACHE_DL_VERSION` on a branch, set the variable on the CUDA job for one PR run,
+  look for CUDA hits in the stats table) and flip the default back when a warm run shows them -- the
+  win was ~51 -> ~15 min with CUDA 13.2.
 
 ### macOS dylib links Homebrew OpenSSL (found by `verify-native-deps.py`)
 

docs/history/llama-cpp-breaking-changes.md

@@ -850,3 +850,13 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r
 | b11495–b11505 | patches + upstream verification | **All ten patches apply unchanged**, verified in order against pristine b11505 with `git apply`. No patch-target file in the range; nothing under `src/`, `common/`, `tools/server/` or `include/` changed, so every drop-check of the b11480 row holds unchanged. |
 | b11505–b11512 | Seven commits, 25 files, ~716/202 lines (79 KiB, the final step). **#26004 keeps the context checkpoints across a slot save and restore**: the slot file gains an appendix after the llama state payload (magic `SCKP`, version 1, a count, then per checkpoint `n_tokens`/`pos_min`/`pos_max` and the target, draft and speculative blobs), counted in the `n_written` / `n_read` of the save and restore results; on restore it is read bounds-checked -- a missing, truncated or foreign appendix is ignored with a warning, never fatal, so a slot file written before this restores as before -- and one draft checkpoint is test-loaded, the draft data dropped when it does not fit the current draft context, where `common_prompt_checkpoint::load_dft()` used to `GGML_ABORT` on a size mismatch (`load_tgt`/`load_dft` return `bool` now, `common/common.h`). `LlamaModel.handleSlotAction(SAVE/RESTORE)` carries the same payload; the response keys are unchanged. **#30112 splits the MoE expert cache across several GPUs** like the layers (`--tensor-split`; the `--moe-cache-mib` help text, `common_params`, `llama-moe-cache.cpp`) -- `setMoeCacheMib`'s Javadoc already says so. **CUDA:** MMQ out-of-bounds reads (#29953), a looped PAD kernel beyond 65535 rows or slices (#30147), and the CCCL version guard in `argsort.cu` compares the packed `CCCL_VERSION` (#29453: the per-component compare turned `STRIDED_ITERATOR_AVAILABLE` off on a major rollover -- with the pinned CCCL v3.4.3 it is true before and after, so no change for the CUDA jars until CCCL 4). #30111 reads DFlash's tied output weights from the GGUF metadata; #29668 WebUI settings on the first visit in router mode (auto-followed). **No project source change.** |
 | b11505–b11512 | patches + upstream verification | **All ten patches apply unchanged**, verified in order against pristine b11512 with `git apply` and by a fresh configure through the real `FetchContent` path. Patch-target files in the range: `common/arg.cpp` (`0001`, `0015`: a help-text change only) and `tools/server/server-context.cpp` (`0002`, `0003`: the checkpoint appendix helpers are private members added away from `load_model` and the similarity getter/setter). Drop-checks against pristine b11512 -- the final state of the bump: `common_params_parse` still carries the `#ifdef _WIN32` `argv = utf8.ptrs.data()` override and `common_params_parse_main` is absent (`0001`); the load-progress assignment is unconditional (`0002`); no `slot_prompt_similarity` getter (`0003`); no embedded-mode, attach or worker-command counterpart (`0006`-`0008`); no `split_sum == 0` guard (`0012`); `common/log.h` names only `common_log_default_callback`, no sink (`0014`); no `stop_server` and the connect failure still aborts in `ggml-rpc.cpp` (`0015`); no `kolibri` under `src/` (`0016`). No `.github/` change upstream in the whole range. |
+| b11512–b11516 | Four commits, 11 files, ~295/167 lines (40 KiB, one step), backends only. **CUDA: top-k algorithm selection reworked** (#28713): a grid-over-rows radix select replaces CUB's per-row `DeviceTopKKernel` once a batch has `GGML_CUDA_TOPK_RADIX_MIN_ROWS` rows, bitonic stays for short rows that fit one wave of blocks, DeviceTopK or the CUB argsort serves a single long row, and radix select works in chunks so its scratch memory stays bounded -- upstream measured top-k on a 34,816-token MoE prompt from 5,762 ms to 942 ms; the thresholds stay build-time overridable, HIP and MUSA keep theirs. **SYCL: Q5_K reorder-layout MMVQ and a fused GLU** (#29375; the two Linux SYCL jars and the Windows one). A MUSA FWHT fix (#30167) and a Hexagon IM2COL DMA-ring overflow (#30189), neither built here. **No project source change.** |
+| b11512–b11516 | patches + upstream verification | **All eleven patches apply unchanged**, verified in order against pristine b11516 with `git apply`. No patch-target file in the range; nothing under `src/`, `common/`, `tools/server/`, `include/` or `ggml/src/ggml-cpu/` changed, so every drop-check of the b11505–b11512 row holds unchanged -- and `patches/0017`'s own check, the four uncast `_mm_prefetch` calls in `ggml/src/ggml-cpu/arch/x86/quants.c` (lines 782/783 and 800/801), still finds them uncast. |
+| b11516–b11517 | One commit, 4 files, ~223/225 lines (81 KiB, one step). **CUDA MMQ: the host-side config helpers take the `src1` precision** (#30168; `mmq.cu`, `mmq.cuh`, `mmq-load-tiles.cuh`, `mmq-vec-dot.cuh` -- a signature change threaded through the tile loaders and vector dot products, which is why a one-line idea weighs 81 KiB). Inside the CUDA backend, which `jllama` builds as a whole from upstream's sources; nothing of it is a public API. **No project source change.** |
+| b11516–b11517 | patches + upstream verification | **All eleven patches apply unchanged**, verified in order against pristine b11517 with `git apply`. No patch-target file in the range; nothing outside `ggml/src/ggml-cuda/` changed, so every drop-check of the b11505–b11512 row holds unchanged. |
+| b11517–b11518 | One commit, 17 files, ~39/4 lines (32 KiB, one step). **Metal: flash-attention kernels for the 128/96 head-size pair, and every compiled (K, V) pair is accepted** (#30209; `ggml-metal-device.m` decides `supports_op` from the compiled kernel set instead of a fixed list, `ggml-metal-ops.cpp`, `-tuning.cpp`, and one instantiation line in each of the 14 `fa_*.metal` / `fa_vec_*.metal` kernel files). The shipped `metal-macos-aarch64` jar embeds the Metal library (`LLAMA_METAL_EMBED_LIBRARY=ON`), so a model with that head-size pair now runs flash attention on the GPU where it fell back before. **No project source change.** |
+| b11517–b11518 | patches + upstream verification | **All eleven patches apply unchanged**, verified in order against pristine b11518 with `git apply`. No patch-target file in the range; nothing outside `ggml/src/ggml-metal/` changed, so every drop-check of the b11505–b11512 row holds unchanged. |
+| b11518–b11521 | Three commits, 214 files, ~8430/2907 lines (530 KiB, one step -- no intermediate tag between b11518 and b11521). **#30159 changes the server's default port from 8080 to 9931** (`common_params::port` in `common/common.h`, `server_http_context::port` in `tools/server/server-http.h`, the `--server-base` help text in `common/arg.cpp`; the "default port will be changed to :9931" notice `llama_server()` printed since #26508 is removed from `tools/server/server.cpp`; the READMEs, bench scripts and Dockerfiles follow). It reaches this project through `NativeServer`: an argv without `--port` now binds 9931, in classic mode and in attach mode alike (`llama_server_attach` parses its HTTP argv through `common_params_parse`), so **`NativeServer.DEFAULT_PORT` -- what `getPort()` reports when no port flag is given -- moves to 9931** with its Javadoc and `NativeServerSmokeTest`. `OpenAiCompatServer` keeps its own `DEFAULT_PORT = 8080`: it is the Java transport with no upstream coupling, and the constant is a public API of this project. Every README example passes `--port` explicitly and is unaffected. **#29583 adds the models manager to the WebUI** (~200 files under `tools/ui/`: drawer and sheet overlays, a models store with a download-status feed, Hugging Face and migration services, the selector regrouped by provider, the manager table with repo/quant/status rows, `vaul-svelte` and a newer `@lucide/svelte` in `package.json`). `build-webui` auto-follows the tag (`npm ci && npm run build`, `scripts/ui-assets.cmake` unchanged), and the routes the manager drives are the router's model-management endpoints the embedded server already carries. #30212 is upstream's own HIP CI quality check (`scripts/hip/gcn-cdna-vgpr-check.py`). **Project source change: `NativeServer.DEFAULT_PORT` and its test; CHANGELOG.** |
+| b11518–b11521 | patches + upstream verification | **All eleven patches apply unchanged**, verified in order against pristine b11521 with `git apply`. Patch-target files in the range: `common/arg.cpp` (`0001`, `0015`: one help string of `--server-base`, away from `common_params_parse` and `add_rpc_devices`) and `tools/server/server.cpp` (`0006`, `0007`: the eight removed lines sat after the `listening on` loop of `llama_server()`, between `0006`'s embedded-mode parse hunk and the router branch, and outside the route table `0007` factors out -- both hunks apply with their context intact). Drop-checks unchanged: `common_params_parse` still carries the Windows `argv` override, the load-progress assignment is still unconditional, no embedded-mode or attach counterpart upstream. |
+| b11521–b11529 | Eight commits, 22 files, ~311/187 lines (49 KiB, the final step). **ggml: `ggml_acc` out-of-bounds write with a large offset** (#30135, `ggml/src/ggml.c`): the `size_t` offset was narrowed to `int32_t` unchecked, a value past `INT32_MAX` became negative, the forward sign-extended it and the bounds assertion wrapped, so the write landed below the destination buffer; the offset is now checked before the narrowing, as `ggml_set_impl` already did -- core code every natives jar builds. **Vulkan: `rms_norm` workgroup-count overflow** (#30145; `ggml-vulkan.cpp`, `rms_norm.comp`) for the three Vulkan jars. **Meta backend handles host views** (#30217): `--split-mode tensor` with partial offload asserted on KV-cache views left inside a meta split; such nodes are accepted when they are views of host tensors, and `LLM_ARCH_K2_HORIZON` leaves the list of architectures `llm_arch_supports_sm_tensor()` refuses (`src/llama-arch.cpp`, one line) -- relevant to `GpuSplitMode.TENSOR`. MUSA drops `mp_21` from its default architectures (#30203), WebGPU gets 2D workgroup dispatch and `GGML_OP_DUP` (#30219, #30216), the converter reads compressed-tensor mixed-precision NVFP4 checkpoints (#28636, Python), upstream's publish workflow gets a run name (#30214) -- none of it built or run here. **No project source change.** |
+| b11521–b11529 | patches + upstream verification | **All eleven patches apply unchanged**, verified in order against pristine b11529 with `git apply` and by a fresh configure through the real `FetchContent` path. Patch-target file in the range: `src/llama-arch.cpp` (`0016`: the Kolibri-1 entries sit in the arch and tensor tables, the removed line in `llm_arch_supports_sm_tensor()`). Drop-checks against pristine b11529 -- the final state of the bump: `common_params_parse` still carries the `#ifdef _WIN32` `argv = utf8.ptrs.data()` override and `common_params_parse_main` is absent (`0001`); the load-progress assignment is unconditional (`0002`); no `slot_prompt_similarity` getter (`0003`); no embedded-mode, attach or worker-command counterpart (`0006`-`0008`); no `split_sum == 0` guard (`0012`); `common/log.h` names only `common_log_default_callback`, no sink (`0014`); no `stop_server` and the connect failure still aborts in `ggml-rpc.cpp` (`0015`); no `kolibri` under `src/` (`0016`); the four `_mm_prefetch` calls in `quants.c` are still uncast element arithmetic (`0017`). No `.github/` change upstream that concerns this project (only its own `release-publish.yml`). |

llama/CMakeLists.txt

@@ -261,7 +261,7 @@ endif()
 FetchContent_Declare(
 	llama.cpp
 	GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git
-	GIT_TAG        b11512
+	GIT_TAG        b11529
 	PATCH_COMMAND  ${CMAKE_COMMAND}
 		-DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches
 		-DLLAMA_SRC=<SOURCE_DIR>