From 193e46b5392239cc6aa53399ef2e24c1d2cf238b Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 9 Oct 2026 20:14:07 +0000 Subject: [PATCH 1/4] Link BoringSSL statically on every desktop platform; one cpp-httplib copy jllama compiled httplib.cpp itself, without SSL, next to the SSL-enabled copy upstream's llama-common links for downloads. Which copy the linker took decided per platform: Linux jars had no HTTPS at all (an https:// download threw), the macOS dylib depended on Homebrew's openssl@3. Now jllama and jllama_test link upstream's cpp-httplib target, LLAMA_BUILD_BORINGSSL is on for Linux x86-64/aarch64, macOS and Windows (Android and s390x: LLAMA_OPENSSL off), the two Homebrew allowlist lines are gone and crypt32.dll is allowed on Windows. The fat-jar smokes and the macOS smoke gain an HTTPS round: a server behind a self-signed certificate serving a model it downloaded from an https:// URL with the OS certificate store. Verified locally on Linux x86-64: no new NEEDED entry, 603/603 ctest, the Java suite green, TLS 1.3 served and plain HTTP refused on the TLS port, a download refused with an untrusted certificate and completed with SSL_CERT_FILE. macOS and Windows are verified by CI (verify-native-deps, the smokes). Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2 --- .github/buildcheck/nativedeps.py | 18 +++-- .github/smoke-native-macos.sh | 62 +++++++++++++-- .github/smoke-test-fatjar.ps1 | 129 +++++++++++++++++++++--------- .github/smoke-test-fatjar.sh | 130 ++++++++++++++++++++++--------- .github/workflows/publish.yml | 19 ++++- CHANGELOG.md | 19 +++++ CLAUDE.md | 74 ++++++++++++++++-- README.md | 9 ++- TODO.md | 12 --- llama/CMakeLists.txt | 57 ++++++++------ 10 files changed, 394 insertions(+), 135 deletions(-) diff --git a/.github/buildcheck/nativedeps.py b/.github/buildcheck/nativedeps.py index ac9b81764..d7f0b36c3 100755 --- a/.github/buildcheck/nativedeps.py +++ b/.github/buildcheck/nativedeps.py @@ -36,7 +36,11 @@ import sys # The four OS DLLs every Windows build imports, and the dynamic UCRT. -WINDOWS_OS = {"ws2_32.dll", "kernel32.dll", "shell32.dll", "advapi32.dll"} +# crypt32.dll: cpp-httplib loads the Windows certificate store for HTTPS clients +# (CertOpenSystemStoreW, `#pragma comment(lib, "crypt32.lib")` under CPPHTTPLIB_OPENSSL_SUPPORT); an +# OS component since Windows 2000. BoringSSL itself adds no import: its RNG loads bcryptprimitives +# at run time (LoadLibraryW), and everything else is in the static libraries. +WINDOWS_OS = {"ws2_32.dll", "kernel32.dll", "shell32.dll", "advapi32.dll", "crypt32.dll"} # The Universal CRT forwarders. Present because the Windows builds use the HYBRID CRT (static STL + # vcruntime, dynamic UCRT -- llama/CMakeLists.txt explains why), which is what keeps Microsoft's # UCRT security updates reaching a shipped artifact instead of freezing a copy inside jllama.dll. @@ -91,13 +95,11 @@ def ucrt_forwarder(name): "/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate", "/usr/lib/libobjc.A.dylib", "/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation", - "/System/Library/Frameworks/Security.framework/Versions/A/Security", - # KNOWN DEFECT, allowed only so this check reports NEW dependencies: the macOS - # build picks up the runner's Homebrew OpenSSL, so the shipped dylib does not load - # on a Mac without `brew install openssl@3`. See TODO.md ("macOS dylib links - # Homebrew OpenSSL"); remove these two lines with the fix. - "/opt/homebrew/opt/openssl@3/lib/libssl.3.dylib", - "/opt/homebrew/opt/openssl@3/lib/libcrypto.3.dylib"}, + # Security + CoreFoundation: cpp-httplib's system-certificate lookup under + # CPPHTTPLIB_OPENSSL_SUPPORT. The SSL library itself is BoringSSL, linked + # statically (llama/CMakeLists.txt, "HTTPS"); a /opt/homebrew/.../libssl line + # here would mean the build picked up the runner's OpenSSL again. + "/System/Library/Frameworks/Security.framework/Versions/A/Security"}, } # The Visual Studio generator build of the same compiler and runtime. ALLOWED["Windows/x86_64/msvc"] = ALLOWED["Windows/x86_64/cpu"] diff --git a/.github/smoke-native-macos.sh b/.github/smoke-native-macos.sh index f9dc46060..861f51f8d 100755 --- a/.github/smoke-native-macos.sh +++ b/.github/smoke-native-macos.sh @@ -21,18 +21,27 @@ # that never exits, and the assertion that matters here is native-library loadability, not a CLI # exit code. Same job shape, repo-specific assertions. # -# Deliberately model-free: no GGUF, no cache restore, no network — it runs in ~1 min. A full -# model-backed macOS server smoke would be strictly more, but the failure class that actually -# shipped is caught here, so this is the version that is cheap enough to always run. +# No cache restore and no model of the CI set: it runs in ~1 min. A full model-backed macOS server +# smoke would be strictly more, but the failure class that actually shipped is caught here, so this +# is the version that is cheap enough to always run. The one model it touches is the 1 MB +# stories260K.gguf of .github/models.csv, which step 3 downloads itself over HTTPS -- that download +# IS the check: the library links BoringSSL statically (llama/CMakeLists.txt, "HTTPS") and this +# dylib used to depend on the runner's Homebrew OpenSSL, which no check here could see because the +# runner has it. Step 3 proves the HTTPS client against the macOS certificate store and the +# embedded server behind its own TLS certificate, the same round the Linux and Windows rows of +# smoke-fatjar run. # -# Usage: smoke-native-macos.sh +# Usage: smoke-native-macos.sh [port] +# Server output of step 3 is written to server-tls-out.log / server-tls-err.log in the working dir. # directory to search for the jar (recursively) # filename glob; must match EXACTLY ONE jar set -euo pipefail -JAR_DIR="${1:?usage: smoke-native-macos.sh }" -JAR_GLOB="${2:?usage: smoke-native-macos.sh }" +JAR_DIR="${1:?usage: smoke-native-macos.sh [port]}" +JAR_GLOB="${2:?usage: smoke-native-macos.sh [port]}" +TLS_PORT="${3:-18081}" +HTTPS_MODEL_NAME="stories260K.gguf" SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" @@ -91,4 +100,45 @@ echo "== JVM load + JNI round-trip ==" java -cp "$JAR" "$SCRIPT_DIR/smoke/NativeLoadSmoke.java" \ || fail "the packaged native library did not load in a JVM" +# 3) HTTPS in both directions out of the packaged jar: the server behind a self-signed certificate +# (--ssl-key-file / --ssl-cert-file) serving a model it downloaded from an https:// URL, verified +# against the macOS certificate store (Security.framework). -ngl 0: this is about TLS, not Metal. +echo "== HTTPS server + https:// model download ==" +HTTPS_MODEL_URL="$(grep -E "^${HTTPS_MODEL_NAME//./\\.}," "$SCRIPT_DIR/models.csv" | cut -d, -f2- || true)" +[ -n "$HTTPS_MODEL_URL" ] || fail "$HTTPS_MODEL_NAME has no row in $SCRIPT_DIR/models.csv" +openssl req -x509 -newkey rsa:2048 -nodes -keyout "$WORK/tls-key.pem" -out "$WORK/tls-cert.pem" -days 2 \ + -subj "/CN=127.0.0.1" > /dev/null 2>&1 || fail "could not create the self-signed TLS certificate" +export LLAMA_CACHE="$WORK/llama-cache" +mkdir -p "$LLAMA_CACHE" +# -m names where the download lands; with --model-url alone the server starts in ROUTER mode. +java -jar "$JAR" -m "$LLAMA_CACHE/$HTTPS_MODEL_NAME" --model-url "$HTTPS_MODEL_URL" \ + --host 127.0.0.1 --port "$TLS_PORT" --chat-template chatml \ + -ngl 0 --ssl-key-file "$WORK/tls-key.pem" --ssl-cert-file "$WORK/tls-cert.pem" \ + > server-tls-out.log 2> server-tls-err.log & +SERVER_PID=$! +trap 'kill "$SERVER_PID" 2> /dev/null || true; rm -rf "$WORK"' EXIT +CODE="" +for _ in $(seq 1 100); do + if ! kill -0 "$SERVER_PID" 2> /dev/null; then + echo "--- server-tls-out.log ---" && cat server-tls-out.log + echo "--- server-tls-err.log ---" && cat server-tls-err.log + fail "the TLS server exited before becoming healthy (download or load failed)" + fi + CODE="$(curl -sk -o /dev/null -w '%{http_code}' "https://127.0.0.1:$TLS_PORT/health" || true)" + [ "$CODE" = "200" ] && break + sleep 3 +done +if [ "$CODE" != "200" ]; then + echo "--- server-tls-err.log (tail) ---" && tail -50 server-tls-err.log + fail "https://127.0.0.1:$TLS_PORT/health never returned 200 (last code: ${CODE:-none})" +fi +echo "HTTPS health OK" +if curl -s -o /dev/null --max-time 10 "http://127.0.0.1:$TLS_PORT/health"; then + fail "the TLS port answered a plain-HTTP request — the server did not use the certificate" +fi +echo "plain HTTP on the TLS port refused: OK" +find "$LLAMA_CACHE" -type f -name '*.gguf' | grep -q . \ + || fail "no .gguf under $LLAMA_CACHE — the https:// download did not happen" +echo "https:// model download OK: $(find "$LLAMA_CACHE" -type f -name '*.gguf' -exec basename {} \; | head -1)" + echo "smoke test PASSED" diff --git a/.github/smoke-test-fatjar.ps1 b/.github/smoke-test-fatjar.ps1 index 014c4ddfa..2e2bbf76a 100644 --- a/.github/smoke-test-fatjar.ps1 +++ b/.github/smoke-test-fatjar.ps1 @@ -3,11 +3,13 @@ # SPDX-License-Identifier: MIT OR Apache-2.0 # Smoke test for an all-backends server fat jar on a GPU-less Windows runner — -# the PowerShell analogue of smoke-test-fatjar.sh (see there for what it proves). +# the PowerShell analogue of smoke-test-fatjar.sh (see there for what the two rounds prove: +# round 1 plain HTTP with the cached model, round 2 an HTTPS server behind a self-signed +# certificate serving a model it downloaded itself from an https:// URL). # # Usage: smoke-test-fatjar.ps1 -JarDir -JarGlob -Model [-Port

] -# Server output is written to server-out.log / server-err.log in the working dir -# (uploaded by the CI job on failure). +# Server output is written to server-out.log / server-err.log (round 1) and server-tls-out.log / +# server-tls-err.log (round 2) in the working dir (uploaded by the CI job on failure). param( [Parameter(Mandatory = $true)][string]$JarDir, [Parameter(Mandatory = $true)][string]$JarGlob, @@ -15,9 +17,11 @@ param( [int]$Port = 18080 ) $ErrorActionPreference = 'Stop' +$TlsPort = $Port + 1 +$HttpsModelName = 'stories260K.gguf' -function Dump-ServerLogs { - foreach ($log in 'server-out.log', 'server-err.log') { +function Dump-ServerLogs([string[]]$Logs) { + foreach ($log in $Logs) { if (Test-Path $log) { Write-Host "--- $log (tail) ---" Get-Content $log -Tail 50 @@ -25,58 +29,109 @@ function Dump-ServerLogs { } } -$jars = @(Get-ChildItem -Path $JarDir -Filter $JarGlob -File) -if ($jars.Count -ne 1) { - Write-Error "expected exactly 1 jar matching $JarGlob in $JarDir, got $($jars.Count)" -} -$jar = $jars[0].FullName -if (-not (Test-Path $Model)) { Write-Error "model file missing: $Model" } -Write-Host "smoke jar: $jar" - -$proc = Start-Process java -PassThru -NoNewWindow ` - -RedirectStandardOutput server-out.log -RedirectStandardError server-err.log ` - -ArgumentList '-jar', $jar, '-m', $Model, '--host', '127.0.0.1', '--port', "$Port", '--chat-template', 'chatml' -try { - # Poll /health until 200 (model loaded); 100 x 3 s = 5 min budget. An early server - # exit (e.g. an UnsatisfiedLinkError the fallback chain failed to absorb) fails fast. - $healthy = $false +# Poll /health until 200 (model loaded); 100 x 3 s = 5 min budget. An early server exit (e.g. an +# UnsatisfiedLinkError the fallback chain failed to absorb) fails fast. -SkipCertificateCheck +# accepts round 2's self-signed certificate and is a no-op over plain HTTP. +function Wait-Healthy($Proc, [string]$Base, [string[]]$Logs) { foreach ($i in 1..100) { - if ($proc.HasExited) { - Dump-ServerLogs - Write-Error "server process exited before becoming healthy (exit code $($proc.ExitCode))" + if ($Proc.HasExited) { + Dump-ServerLogs $Logs + Write-Error "server process exited before becoming healthy ($Base, exit code $($Proc.ExitCode))" } try { - $health = Invoke-WebRequest -Uri "http://127.0.0.1:$Port/health" -UseBasicParsing -TimeoutSec 5 - if ($health.StatusCode -eq 200) { $healthy = $true; break } + $health = Invoke-WebRequest -Uri "$Base/health" -UseBasicParsing -SkipCertificateCheck -TimeoutSec 5 + if ($health.StatusCode -eq 200) { return } } catch { # 503 while loading / connection refused before listening: keep polling. } Start-Sleep -Seconds 3 } - if (-not $healthy) { - Dump-ServerLogs - Write-Error "/health never returned 200" - } - Write-Host "health OK" + Dump-ServerLogs $Logs + Write-Error "$Base/health never returned 200" +} +# A chat completion with one valid choice, over $Base. +function Test-ChatCompletion([string]$Base) { $body = '{"messages":[{"role":"user","content":"Say hello."}],"max_tokens":16,"temperature":0}' - $response = Invoke-RestMethod -Uri "http://127.0.0.1:$Port/v1/chat/completions" ` + $response = Invoke-RestMethod -Uri "$Base/v1/chat/completions" -SkipCertificateCheck ` -Method Post -ContentType 'application/json' -Body $body -TimeoutSec 300 if (-not $response.choices -or $response.choices.Count -lt 1 -or -not $response.choices[0].message) { Write-Error "malformed chat completion response: $($response | ConvertTo-Json -Depth 6 -Compress)" } - Write-Host "chat completion OK: $($response.choices[0].message.content)" + Write-Host "chat completion OK ($Base): $($response.choices[0].message.content)" +} + +$jars = @(Get-ChildItem -Path $JarDir -Filter $JarGlob -File) +if ($jars.Count -ne 1) { + Write-Error "expected exactly 1 jar matching $JarGlob in $JarDir, got $($jars.Count)" +} +$jar = $jars[0].FullName +if (-not (Test-Path $Model)) { Write-Error "model file missing: $Model" } +Write-Host "smoke jar: $jar" + +$modelsCsv = Join-Path $PSScriptRoot 'models.csv' +$httpsModelUrl = (Get-Content $modelsCsv | Where-Object { $_ -like "$HttpsModelName,*" } | Select-Object -First 1) +if (-not $httpsModelUrl) { Write-Error "$HttpsModelName has no row in $modelsCsv (round 2 downloads it over HTTPS)" } +$httpsModelUrl = $httpsModelUrl.Substring($HttpsModelName.Length + 1) + +# ---- Round 1: plain HTTP, the cached model ------------------------------------------------------- +$proc = Start-Process java -PassThru -NoNewWindow ` + -RedirectStandardOutput server-out.log -RedirectStandardError server-err.log ` + -ArgumentList '-jar', $jar, '-m', $Model, '--host', '127.0.0.1', '--port', "$Port", '--chat-template', 'chatml' +try { + Wait-Healthy $proc "http://127.0.0.1:$Port" @('server-out.log', 'server-err.log') + Write-Host "health OK" + Test-ChatCompletion "http://127.0.0.1:$Port" # The loader must have reported its backend decision (a chosen backend on a GPU - # machine, the CPU fallback on a GPU-less runner) — this pins that the manifest was - # actually read, i.e. the smoke really ran the multi-backend code path. + # machine, the CPU fallback on a GPU-less runner) — this pins that the smoke really + # ran the multi-backend code path. $selection = Select-String -Path 'server-out.log', 'server-err.log' ` - -Pattern '\[jllama\] (using native backend|no manifest backend loadable)' + -Pattern '\[jllama\] using native backend' if (-not $selection) { - Dump-ServerLogs - Write-Error "no backend-selection log line found - the backend manifest was not processed" + Dump-ServerLogs @('server-out.log', 'server-err.log') + Write-Error "no backend-selection log line found - the loader did not report a backend" } Write-Host "backend selection: $($selection[0].Line)" +} finally { + if (-not $proc.HasExited) { Stop-Process -Id $proc.Id -Force } +} + +# ---- Round 2: HTTPS server + https:// model download --------------------------------------------- +# openssl comes with Git for Windows (usr\bin is not on PATH); the runner images have it. +$openssl = (Get-Command openssl -ErrorAction SilentlyContinue).Source +if (-not $openssl) { $openssl = Join-Path $env:ProgramFiles 'Git\usr\bin\openssl.exe' } +if (-not (Test-Path $openssl)) { Write-Error "no openssl CLI found to mint the TLS test certificate" } +& $openssl req -x509 -newkey rsa:2048 -nodes -keyout tls-key.pem -out tls-cert.pem -days 2 -subj '/CN=127.0.0.1' 2>$null +if ($LASTEXITCODE -ne 0 -or -not (Test-Path tls-cert.pem)) { Write-Error "could not create the self-signed TLS certificate" } +$env:LLAMA_CACHE = Join-Path (Get-Location) 'llama-cache' +if (Test-Path $env:LLAMA_CACHE) { Remove-Item -Recurse -Force $env:LLAMA_CACHE } +New-Item -ItemType Directory -Path $env:LLAMA_CACHE | Out-Null + +# -m names where the download lands; with --model-url alone the server starts in ROUTER mode. +$proc = Start-Process java -PassThru -NoNewWindow ` + -RedirectStandardOutput server-tls-out.log -RedirectStandardError server-tls-err.log ` + -ArgumentList '-jar', $jar, '-m', (Join-Path $env:LLAMA_CACHE $HttpsModelName), '--model-url', $httpsModelUrl, ` + '--host', '127.0.0.1', '--port', "$TlsPort", ` + '--chat-template', 'chatml', '--ssl-key-file', 'tls-key.pem', '--ssl-cert-file', 'tls-cert.pem' +try { + Wait-Healthy $proc "https://127.0.0.1:$TlsPort" @('server-tls-out.log', 'server-tls-err.log') + Write-Host "HTTPS health OK" + Test-ChatCompletion "https://127.0.0.1:$TlsPort" + # The port must really speak TLS: a plain-HTTP request to it fails. + $plainAnswered = $false + try { + Invoke-WebRequest -Uri "http://127.0.0.1:$TlsPort/health" -UseBasicParsing -TimeoutSec 10 | Out-Null + $plainAnswered = $true + } catch { + # expected: the TLS server rejects the plain request + } + if ($plainAnswered) { Write-Error "the TLS port answered a plain-HTTP request - the server did not use the certificate" } + Write-Host "plain HTTP on the TLS port refused: OK" + # The model came over HTTPS into this run's cache, not from the runner's model directory. + $downloaded = @(Get-ChildItem -Path $env:LLAMA_CACHE -Recurse -File -Filter '*.gguf') + if ($downloaded.Count -lt 1) { Write-Error "no .gguf under $env:LLAMA_CACHE - the https:// download did not happen" } + Write-Host "https:// model download OK: $($downloaded[0].Name)" Write-Host "smoke test PASSED" } finally { if (-not $proc.HasExited) { Stop-Process -Id $proc.Id -Force } diff --git a/.github/smoke-test-fatjar.sh b/.github/smoke-test-fatjar.sh index 9a2da9136..09fc5857f 100755 --- a/.github/smoke-test-fatjar.sh +++ b/.github/smoke-test-fatjar.sh @@ -4,22 +4,35 @@ # # SPDX-License-Identifier: MIT OR Apache-2.0 -# Smoke test for an all-backends server fat jar on a GPU-less runner: -# `java -jar` must start the embedded server — with every GPU backend failing its -# load cleanly and the loader falling back to the CPU backend — then answer GET -# /health with 200 and a POST /v1/chat/completions with a valid choice. This -# exercises backend probing, per-backend extraction, and the fallback chain -# end-to-end through a real fat-jar launch. +# Smoke test for an all-backends server fat jar on a GPU-less runner, in two rounds. +# +# Round 1 (plain HTTP): `java -jar` must start the embedded server — with every GPU backend +# failing its load cleanly and the loader falling back to the CPU backend — then answer GET +# /health with 200 and a POST /v1/chat/completions with a valid choice. This exercises backend +# probing, per-backend extraction, and the fallback chain end-to-end through a real fat-jar launch. +# +# Round 2 (HTTPS, both directions): the library links BoringSSL statically (llama/CMakeLists.txt, +# "HTTPS"), so out of the release asset two things must work on every platform -- the embedded +# server behind its own TLS key and certificate (--ssl-key-file / --ssl-cert-file), and a model +# download from an https:// URL verified against the OS certificate store (crypt32 on Windows, +# Security on macOS, /etc/ssl on Linux). Neither held before 5.2.0: the Linux jars had no SSL at +# all (an https:// download threw "HTTPS is not supported"), and macOS depended on the runner's +# Homebrew OpenSSL. The download is the 1 MB stories260K.gguf of .github/models.csv, into a cache +# directory of this run (LLAMA_CACHE), so round 2 also proves the HTTPS client path with a real +# certificate chain, which no local check can. # # Usage: smoke-test-fatjar.sh [port] -# Server output is written to server-out.log / server-err.log in the working dir -# (uploaded by the CI job on failure). +# Server output is written to server-out.log / server-err.log (round 1) and server-tls-out.log / +# server-tls-err.log (round 2) in the working dir (uploaded by the CI job on failure). set -euo pipefail JAR_DIR="${1:?usage: smoke-test-fatjar.sh [port]}" JAR_GLOB="${2:?usage: smoke-test-fatjar.sh [port]}" MODEL="${3:?usage: smoke-test-fatjar.sh [port]}" PORT="${4:-18080}" +TLS_PORT=$((PORT + 1)) +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +HTTPS_MODEL_NAME="stories260K.gguf" fail() { echo "::error::$*" >&2 @@ -32,48 +45,91 @@ JAR="${JARS[0]}" [ -f "$MODEL" ] || fail "model file missing: $MODEL" echo "smoke jar: $JAR" -java -jar "$JAR" -m "$MODEL" --host 127.0.0.1 --port "$PORT" --chat-template chatml \ - > server-out.log 2> server-err.log & -SERVER_PID=$! -cleanup() { kill "$SERVER_PID" 2> /dev/null || true; } +HTTPS_MODEL_URL="$(grep -E "^${HTTPS_MODEL_NAME//./\\.}," "$SCRIPT_DIR/models.csv" | cut -d, -f2- || true)" +[ -n "$HTTPS_MODEL_URL" ] || fail "$HTTPS_MODEL_NAME has no row in $SCRIPT_DIR/models.csv (round 2 downloads it over HTTPS)" + +SERVER_PID="" +cleanup() { [ -n "$SERVER_PID" ] && kill "$SERVER_PID" 2> /dev/null || true; } trap cleanup EXIT -# Poll /health until 200 (model loaded); 100 x 3 s = 5 min budget. An early server -# exit (e.g. an UnsatisfiedLinkError the fallback chain failed to absorb) fails fast. -CODE="" -for _ in $(seq 1 100); do - if ! kill -0 "$SERVER_PID" 2> /dev/null; then - echo "--- server-out.log ---" && cat server-out.log - echo "--- server-err.log ---" && cat server-err.log - fail "server process exited before becoming healthy" - fi - CODE="$(curl -s -o /dev/null -w '%{http_code}' "http://127.0.0.1:$PORT/health" || true)" - [ "$CODE" = "200" ] && break - sleep 3 -done -if [ "$CODE" != "200" ]; then - echo "--- server-out.log (tail) ---" && tail -50 server-out.log - echo "--- server-err.log (tail) ---" && tail -50 server-err.log - fail "/health never returned 200 (last code: ${CODE:-none})" -fi -echo "health OK" +# Poll /health until 200 (model loaded); 100 x 3 s = 5 min budget. An early server exit (e.g. an +# UnsatisfiedLinkError the fallback chain failed to absorb) fails fast. $1 is the base URL, $2/$3 +# the server's stdout/stderr logs. +wait_healthy() { + local base="$1" out="$2" err="$3" code="" + for _ in $(seq 1 100); do + if ! kill -0 "$SERVER_PID" 2> /dev/null; then + echo "--- $out ---" && cat "$out" + echo "--- $err ---" && cat "$err" + fail "server process exited before becoming healthy ($base)" + fi + code="$(curl -sk -o /dev/null -w '%{http_code}' "$base/health" || true)" + [ "$code" = "200" ] && return 0 + sleep 3 + done + echo "--- $out (tail) ---" && tail -50 "$out" + echo "--- $err (tail) ---" && tail -50 "$err" + fail "$base/health never returned 200 (last code: ${code:-none})" +} -RESPONSE="$(curl -sS --fail -X POST "http://127.0.0.1:$PORT/v1/chat/completions" \ - -H 'Content-Type: application/json' \ - -d '{"messages":[{"role":"user","content":"Say hello."}],"max_tokens":16,"temperature":0}')" \ - || fail "chat completion request failed" -echo "$RESPONSE" | python3 -c ' +# A chat completion with one valid choice, over $1 (the base URL). `-k` accepts the self-signed +# certificate of round 2 and is a no-op over plain HTTP. +chat_completion() { + local base="$1" response + response="$(curl -sSk --fail -X POST "$base/v1/chat/completions" \ + -H 'Content-Type: application/json' \ + -d '{"messages":[{"role":"user","content":"Say hello."}],"max_tokens":16,"temperature":0}')" \ + || fail "chat completion request failed ($base)" + echo "$response" | python3 -c ' import json, sys response = json.load(sys.stdin) message = response["choices"][0]["message"] assert message is not None, "choices[0].message missing" print("chat completion OK:", json.dumps(message)[:200]) -' || fail "malformed chat completion response: $RESPONSE" +' || fail "malformed chat completion response: $response" +} + +# ---- Round 1: plain HTTP, the cached model ------------------------------------------------------- +java -jar "$JAR" -m "$MODEL" --host 127.0.0.1 --port "$PORT" --chat-template chatml \ + > server-out.log 2> server-err.log & +SERVER_PID=$! +wait_healthy "http://127.0.0.1:$PORT" server-out.log server-err.log +echo "health OK" +chat_completion "http://127.0.0.1:$PORT" # The loader must have reported its backend decision (normally the CPU fallback on a # GPU-less runner; a GPU backend whose runtime happens to be installed may load and # find no device, which is benign) — this pins that the smoke ran the backend probing. grep -hE '\[jllama\] using native backend' server-out.log server-err.log \ || fail "no backend-selection log line found — the loader did not report a backend" +kill "$SERVER_PID" 2> /dev/null || true +wait "$SERVER_PID" 2> /dev/null || true +SERVER_PID="" + +# ---- Round 2: HTTPS server + https:// model download --------------------------------------------- +command -v openssl > /dev/null || fail "the openssl CLI is needed to mint the TLS test certificate" +openssl req -x509 -newkey rsa:2048 -nodes -keyout tls-key.pem -out tls-cert.pem -days 2 \ + -subj "/CN=127.0.0.1" > /dev/null 2>&1 || fail "could not create the self-signed TLS certificate" +# -m names where the download lands; with --model-url alone the server starts in ROUTER mode +# (upstream decides on model.path / hf_repo / docker_repo before the URL is resolved). +export LLAMA_CACHE="$PWD/llama-cache" +rm -rf "$LLAMA_CACHE" && mkdir -p "$LLAMA_CACHE" +java -jar "$JAR" -m "$LLAMA_CACHE/$HTTPS_MODEL_NAME" --model-url "$HTTPS_MODEL_URL" \ + --host 127.0.0.1 --port "$TLS_PORT" --chat-template chatml \ + --ssl-key-file tls-key.pem --ssl-cert-file tls-cert.pem \ + > server-tls-out.log 2> server-tls-err.log & +SERVER_PID=$! +wait_healthy "https://127.0.0.1:$TLS_PORT" server-tls-out.log server-tls-err.log +echo "HTTPS health OK" +chat_completion "https://127.0.0.1:$TLS_PORT" +# The port must really speak TLS: a plain-HTTP request to it is refused (curl exits non-zero). +if curl -s -o /dev/null --max-time 10 "http://127.0.0.1:$TLS_PORT/health"; then + fail "the TLS port answered a plain-HTTP request — the server did not use the certificate" +fi +echo "plain HTTP on the TLS port refused: OK" +# The model came over HTTPS into this run's cache, not from the runner's model directory. +find "$LLAMA_CACHE" -type f -name '*.gguf' | grep -q . \ + || fail "no .gguf under $LLAMA_CACHE — the https:// download did not happen" +echo "https:// model download OK: $(find "$LLAMA_CACHE" -type f -name '*.gguf' -exec basename {} \; | head -1)" echo "smoke test PASSED" diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 9b9676679..3e6edd497 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -2696,6 +2696,8 @@ jobs: path: | server-out.log server-err.log + server-tls-out.log + server-tls-err.log rpc-server.log rpc-client-out.log rpc-client-err.log @@ -2759,9 +2761,11 @@ jobs: # no `all-macos-*` fat jar to target: macOS has no GPU classifier (Metal ships in the default # jar), so package-fatjars builds no macOS variant. And it asserts native loadability rather than # a CLI exit code — this jar's Main-Class is a server that never returns, and `codesign --strict` - # is what actually detects a dylib assembled from two builds. Deliberately model-free (no GGUF, - # no cache restore, ~1 min): a full model-backed macOS server smoke would be strictly more, but - # this catches the failure class that shipped and is cheap enough to always run. + # is what actually detects a dylib assembled from two builds. No cache restore (~1 min): a full + # model-backed macOS server smoke would be strictly more, but this catches the failure class that + # shipped and is cheap enough to always run. Its last step downloads the 1 MB stories260K.gguf + # over HTTPS and serves it behind a self-signed certificate -- the proof that the dylib's SSL is + # the statically linked BoringSSL and not the runner's Homebrew OpenSSL, which it used to be. # --------------------------------------------------------------------------- smoke-fatjar-macos: @@ -2780,6 +2784,15 @@ jobs: java-version-file: .java-version - name: Run packaged-native smoke test run: .github/smoke-native-macos.sh jars 'llama-*-jar-with-dependencies.jar' + - name: Upload server logs + if: failure() + uses: actions/upload-artifact@v7 + with: + name: fatjar-smoke-macos-logs + path: | + server-tls-out.log + server-tls-err.log + if-no-files-found: warn report: name: Report diff --git a/CHANGELOG.md b/CHANGELOG.md index 22a53ce91..15506f754 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,12 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by ## [Unreleased] ### Added +- **CI proves HTTPS in both directions on the release assets.** The fat-jar smokes (Linux x86-64 / + aarch64, Windows x86-64 / arm64) and the macOS smoke each start the server behind a self-signed + certificate (`--ssl-key-file` / `--ssl-cert-file`; a plain-HTTP request to the port must be refused) + serving a model it downloaded itself from an `https://` URL (the 1 MB `stories260K.gguf` of + `models.csv`) with the operating system's certificate store -- the check that would have caught the + Linux defect below, and the only one that can see the macOS one (the runner has Homebrew's OpenSSL). - **`examples/jbang/Chat.java`**, a one-file console chat that [JBang](https://www.jbang.dev) runs without a checkout or a build (`jbang https://github.com/bernardladenthin/java-llama.cpp/blob/main/examples/jbang/Chat.java model.gguf`). Its `//DEPS` lines name the classes jar and the CPU natives jar of every desktop platform @@ -275,6 +281,19 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by 4.4.7 (none of the files the agent's carried JLine fixes touch changed between 4.4.6 and 4.4.7). ### Fixed +- **HTTPS works out of every desktop natives jar, and the macOS dylib no longer needs Homebrew's OpenSSL.** + `jllama` compiled cpp-httplib itself, without SSL, next to the SSL-enabled copy upstream's `llama-common` + (the download code) links -- two incompatible copies of the same classes in one link, and which one the + linker took was decided per platform: on Linux `--model-url https://…` and `-hf` threw + "HTTPS is not supported", and on macOS the dylib came to depend on `/opt/homebrew/opt/openssl@3` + and failed to load on a Mac without that formula (`verify-native-deps.py` carried the two paths as a + known defect). Now `jllama` links upstream's `cpp-httplib` target, and BoringSSL is built from source + and linked statically on Linux x86-64 / aarch64, macOS and Windows (where it already was), as + upstream's own release builds do; Android and s390x stay without SSL (`LLAMA_OPENSSL=OFF`). The + embedded server's `--ssl-key-file` / `--ssl-cert-file` work as a result. Certificates are verified + against the OS store: crypt32 on Windows (the one new import of `jllama.dll`, allowlisted), + Security.framework on macOS, `/etc/ssl/certs` and `/etc/ssl/cert.pem` on Linux (`SSL_CERT_FILE` / + `SSL_CERT_DIR` override). The library grows by about 2.5 MB, the code of BoringSSL's two static libraries. - **`LlamaLoader` prints its two diagnostic lines to stderr**, `[jllama] using native backend '…'` and `[jllama] extracted '…'`, where they went to stdout before. A router worker JVM (`NativeServer.setWorkerCommand`) printed them onto the router's command pipe, which since llama.cpp b11401 diff --git a/CLAUDE.md b/CLAUDE.md index 31a9477c0..330eefbe7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1201,10 +1201,10 @@ commands — and holds every CPU directory (`cpu`, `metal`, `msvc`) and the Andr exact per-directory allowlist (the other GPU directories only to a denylist, since they need their vendor runtime). A new CPU directory without an allowlist fails too. (That every build arrived is the merge step's check against -`natives.csv`.) **It found a pre-existing defect on its first run**: the macOS dylib links Homebrew's -`openssl@3` (`/opt/homebrew/opt/openssl@3/lib/libssl.3.dylib` + `libcrypto`), so it does not load on a -Mac without that formula. Those two paths are in the allowlist marked as a known defect so the -check reports only *new* dependencies; the fix is on file in `TODO.md`. +`natives.csv`.) **It found a pre-existing defect on its first run**: the macOS dylib linked Homebrew's +`openssl@3` (`/opt/homebrew/opt/openssl@3/lib/libssl.3.dylib` + `libcrypto`), so it did not load on a +Mac without that formula. Fixed in 5.2.0 by linking BoringSSL statically (see "HTTPS: BoringSSL, +statically" below); the allowlist no longer carries those paths, so their return reds `package`. **Three places every argv goes through (`src/main/cpp/rpc_support.hpp`).** ggml's backend registry is process-wide and has no unregister, so an RPC server registered by one `--rpc` load stays a @@ -1292,6 +1292,70 @@ block), and upstream's own release binaries, which all build with `GGML_RPC=ON`, no error path for it); Android needs the app's `INTERNET` permission even on loopback, which the AAR deliberately does not request; the server serves one client at a time; no authentication or TLS. +## HTTPS: BoringSSL, statically + +Every desktop natives jar can fetch a model from an `https://` URL (`--model-url`, `-hf`, the +`ModelParameters` setters behind them) and serve the embedded server behind its own TLS certificate +(`--ssl-key-file` / `--ssl-cert-file`). The SSL library is **BoringSSL, built from source and linked +statically** -- `LLAMA_BUILD_BORINGSSL ON` in `llama/CMakeLists.txt` (the "HTTPS" block), the flag +upstream's own release builds pass; upstream's `vendor/cpp-httplib/CMakeLists.txt` fetches the pinned +tag and links `ssl`/`crypto` into its `cpp-httplib` target. **Android and s390x are the exceptions** +(`LLAMA_OPENSSL=OFF`, so `find_package` cannot link a system OpenSSL by accident): the dockcross +cross-clang is not a configuration BoringSSL is built for here, and big-endian is outside its +supported set. `verify-native-deps.py` is what holds the line: a `libssl` of any kind in a shipped +library's dependency list fails `package`. The one import this adds is Windows' `crypt32.dll` +(cpp-httplib's certificate-store lookup), an OS component since Windows 2000, in `WINDOWS_OS`. + +**The defect this replaced was a link-order lottery, not a missing flag.** Until 5.2.0 `jllama` +compiled `vendor/cpp-httplib/httplib.cpp` itself, deliberately without `CPPHTTPLIB_OPENSSL_SUPPORT` +(the embedded server was "plain HTTP, front it with a TLS proxy"), while upstream's `llama-common` +-- `common/download.cpp` -- linked its own `cpp-httplib` static library *with* the define whenever an +OpenSSL was found. Two copies of the same classes with different layouts in one link; the linker's +choice per platform decided the behaviour: on **Linux** our objects won, `libssl` was never pulled in +(measured: no `NEEDED` on it even though CMake had found OpenSSL 3.0.13), and an `https://` download +threw `HTTPS is not supported` -- in **every** Linux jar ever shipped; on **macOS** the archive member +was pulled and the dylib depended on `/opt/homebrew/opt/openssl@3` (found by `verify-native-deps.py`, +carried as a known defect); on **Windows** BoringSSL was already configured but the same duplicate +sat in the link. The fix is one copy: `jllama` and `jllama_test` link upstream's `cpp-httplib` target +(PUBLIC define, so `server-http.cpp` and the download code agree) and no longer compile `httplib.cpp`. + +**Certificates.** cpp-httplib loads the OS store: `crypt32` on Windows, Security.framework on macOS +(the two frameworks were already in the Mac allowlist), and on Linux BoringSSL's compiled-in defaults +`/etc/ssl/certs` (hashed directory) and `/etc/ssl/cert.pem`, overridable with `SSL_CERT_FILE` / +`SSL_CERT_DIR` -- which is also how a private CA or a corporate proxy's certificate is trusted. +Upstream's download code has no `--insecure` and no proxy support; this project adds neither. + +**Measured locally (Linux x86-64, the sandbox, BoringSSL `0.20260929.0`)** before the CI run: the +same `NEEDED` list as before (no `libssl`, no `libcrypto`), 56 `httplib::SSLClient` symbols linked, +603/603 C++ tests and the model-free Java suite green; the embedded server in router mode behind a +self-signed certificate answered `/health` and `/models` over TLS 1.3 and refused a plain-HTTP +request on the same port (curl exit 52); a download from a local TLS server was **refused** with +the certificate untrusted (`HEAD failed, status: -1`, nothing written) and **completed** with +`SSL_CERT_FILE` pointing at it (the 64 KB dummy landed at the `-m` path, then failed the GGUF magic, +as intended). Two things learned on the way, both now in the smokes: `--model-url` **alone** starts +the server in **router mode** (`is_router_server` looks at `model.path`, `hf_repo` and `docker_repo` +before the URL is resolved), so a download run names its target with `-m` too; and hostname +verification needs an IP SAN for an IP-literal host, which a bare `CN=127.0.0.1` certificate lacks +-- `curl -k` for the server round, a SAN for the client proof. + +**What CI proves, per platform.** `verify-native-deps.py` (package): the allowlists above. +`smoke-fatjar` (Linux x86-64 / aarch64 in bash, Windows x86-64 / arm64 in PowerShell) and +`smoke-fatjar-macos`: a second server round behind a self-signed certificate, serving a model it +downloaded itself from the `https://` URL of `stories260K.gguf` (1 MB, `models.csv`) into a +`LLAMA_CACHE` of the run -- `/health` and a chat completion over HTTPS, plain HTTP on the TLS port +refused, the `.gguf` present in the cache. That download is the only place besides `download-models` +that reaches Hugging Face, on purpose: it is the end-to-end HTTPS client test with a real +certificate chain and the OS store, which no local check can give. The macOS smoke thereby stopped +being model-free; it is still cache-free and ~1 min. + +**Cost.** BoringSSL is ~400 C/C++ files compiled with `OPENSSL_NO_ASM` (upstream's setting, no +assembler needed), cached by sccache like everything else; the shipped library grows by its +`libssl` + `libcrypto` text. **On a BoringSSL or llama.cpp bump:** the tag is upstream's +(`BORINGSSL_VERSION` in `vendor/cpp-httplib/CMakeLists.txt`); it needs CMake >= 3.22, the floor of +every image here. A local build in a sandbox that cannot reach `boringssl.googlesource.com` passes +`-DBORINGSSL_GIT=https://github.com/google/boringssl` (the mirror carries the tags) or +`-DFETCHCONTENT_SOURCE_DIR_BORINGSSL=`. + ## Qwen3-TTS via `mtmd_helper::gen_audio` (was: OuteTTS build-time extraction) The `TextToSpeech` native pipeline (`tts_engine.{h,cpp}`) drives llama.cpp's upstream Qwen3-TTS @@ -1751,7 +1815,7 @@ If the local check passes (`BUILD SUCCESS`), the `mvn package` job in 2. **`common_json` converts to `std::string` implicitly**, so it binds happily to a `const nlohmann::json &` parameter (via nlohmann's string-constructible converting constructor) and then throws `json::type_error 302` at runtime. Never declare a project helper as taking `nlohmann::json` when callers pass the `json` alias — `require_json_field_impl` is a template for exactly this reason. Other differences to know: no `get_ref`/`array_t`/`type_name()`; a braced list in *value* position does not build an array (write `json::array({...})`); `at(key)` needs an explicit `.get()`; errors are `common_json_error`; and `get()` is limited to the types explicitly specialised in `common/json.cpp`. `log_helpers.hpp` and `train_engine.cpp` keep their own `nlohmann::json` alias — they never touch the server's `json`. - Uses `nlohmann/json` for JSON deserialization of parameters in the two files named above; everything on the server path uses `common_json`. -- The upstream server library (`server-context.cpp`, `server-queue.cpp`, `server-task.cpp`, `server-schema.cpp`, `server-models.cpp`, and — since b9829 — `server-stream.cpp`) is compiled directly into `jllama` via CMake — there is no hand-ported `server.hpp` fork. **`server-stream.cpp` is mandatory, not optional:** it defines the resumable-streaming SSE replay buffer (`g_stream_sessions`, `stream_session_attach_pipe`, `stream_aware_should_stop`, `stream_conv_id_from_headers`, the `stream_pipe_*` types) that `server-context.cpp` / `server-http.cpp` / `server-models.cpp` now `#include "server-stream.h"` and call, so omitting it fails the link with undefined references. It is platform-neutral (threads + std mutex/condvar, no `subprocess.h`/`posix_spawn_*`), so it builds on Android too and sits outside the `server-models.cpp` Android guard. `jllama` wires its own JNI routes and never calls `g_stream_sessions.start_gc()` (only the excluded standalone `server.cpp` `main()` does), so its GC thread stays dormant. **Phase 2:** the upstream HTTP transport (`tools/server/server-http.cpp`) and its `cpp-httplib` backend (`vendor/cpp-httplib/httplib.cpp`) are now compiled into `jllama` too, so the OpenAI-compatible server can be driven natively from JNI *inside* `libjllama` — no separate `llama-server` executable (a JNI shared library loads anywhere a JVM runs, which a standalone binary does not). `server-http.cpp` does `#include "ui.h"` (the WebUI asset table that `tools/ui`/`llama-ui` normally generates); since the Svelte WebUI is not shipped, `src/main/cpp/webui_stub/ui.h` supplies the upstream **empty-asset** interface and leaves `LLAMA_UI_HAS_ASSETS` undefined (all static-asset-serving blocks compile out). `` already resolves through `llama-common` — since upstream #27304 (b10488) not from a `PUBLIC ../vendor` include dir of its own but transitively, via the `vendor::nlohmann` / `vendor::sheredom` INTERFACE targets it links PUBLIC, each of which exports the `vendor/` root (same nlohmann/json 3.12.0 as the FetchContent copy). No SSL: `CPPHTTPLIB_OPENSSL_SUPPORT` is left undefined (plain-HTTP; bind localhost / front with a TLS proxy). **`server.cpp`, `server-tools.cpp` and `server-mcp.cpp` are now compiled in too** (on non-Android — they pull in `subprocess.h`/`posix_spawn_*`, so they share `server-models.cpp`'s Android guard): b9870 exposes `server.cpp`'s entry as `int llama_server(int, char**)` (no `main` in the file), and `patches/0006` makes it embeddable (no process signal handlers, forwarded-argv parse, out-of-band shutdown). **`server-mcp.cpp` is new in b10154** (upstream MCP-server support): both `server.cpp` (`llama_server`'s `mcp_mgr` lifecycle) and `server-tools.cpp` (`tools.setup(..., mcp_mgr)` / `server_mcp::call_tool`) reference `server_mcp`, so it **must** be in the `target_sources` list or the link fails with undefined `server_mcp::{start,shutdown,call_tool,list_tools,~server_mcp}` — **latent on Linux** (a shared object tolerates undefined symbols) but a **hard link error on macOS/ld64 and Windows/MSVC**. It is compiled into `jllama` and — on non-Android — into `jllama_test` too, together with `server.cpp`, `server-tools.cpp`, `server-http.cpp` and `httplib.cpp`, so `test_native_server_shutdown.cpp` can drive the real `llama_server()`. The `NativeServer` JNI bridge (`src/main/cpp/native_server.cpp`) calls `llama_server` on a worker thread, so the **full** upstream server — WebUI and all — runs inside `libjllama`. See "Two server modes" below. +- The upstream server library (`server-context.cpp`, `server-queue.cpp`, `server-task.cpp`, `server-schema.cpp`, `server-models.cpp`, and — since b9829 — `server-stream.cpp`) is compiled directly into `jllama` via CMake — there is no hand-ported `server.hpp` fork. **`server-stream.cpp` is mandatory, not optional:** it defines the resumable-streaming SSE replay buffer (`g_stream_sessions`, `stream_session_attach_pipe`, `stream_aware_should_stop`, `stream_conv_id_from_headers`, the `stream_pipe_*` types) that `server-context.cpp` / `server-http.cpp` / `server-models.cpp` now `#include "server-stream.h"` and call, so omitting it fails the link with undefined references. It is platform-neutral (threads + std mutex/condvar, no `subprocess.h`/`posix_spawn_*`), so it builds on Android too and sits outside the `server-models.cpp` Android guard. `jllama` wires its own JNI routes and never calls `g_stream_sessions.start_gc()` (only the excluded standalone `server.cpp` `main()` does), so its GC thread stays dormant. **Phase 2:** the upstream HTTP transport (`tools/server/server-http.cpp`) is compiled into `jllama` too, and `jllama` links upstream's `cpp-httplib` static target (`vendor/cpp-httplib/`) instead of compiling `httplib.cpp` a second time (it did until 5.2.0, without SSL -- see "HTTPS: BoringSSL, statically" for what that cost), so the OpenAI-compatible server can be driven natively from JNI *inside* `libjllama` — no separate `llama-server` executable (a JNI shared library loads anywhere a JVM runs, which a standalone binary does not). `server-http.cpp` does `#include "ui.h"` (the WebUI asset table that `tools/ui`/`llama-ui` normally generates); since the Svelte WebUI is not shipped, `src/main/cpp/webui_stub/ui.h` supplies the upstream **empty-asset** interface and leaves `LLAMA_UI_HAS_ASSETS` undefined (all static-asset-serving blocks compile out). `` already resolves through `llama-common` — since upstream #27304 (b10488) not from a `PUBLIC ../vendor` include dir of its own but transitively, via the `vendor::nlohmann` / `vendor::sheredom` INTERFACE targets it links PUBLIC, each of which exports the `vendor/` root (same nlohmann/json 3.12.0 as the FetchContent copy). **HTTPS is on**: the `cpp-httplib` target carries `CPPHTTPLIB_OPENSSL_SUPPORT` as a PUBLIC define whenever an SSL library is configured, and BoringSSL is linked statically on every desktop platform, so `server-http.cpp` serves TLS with `--ssl-key-file`/`--ssl-cert-file` and `common/download.cpp` fetches `https://` URLs (not on Android and s390x, where `LLAMA_OPENSSL=OFF`). **`server.cpp`, `server-tools.cpp` and `server-mcp.cpp` are now compiled in too** (on non-Android — they pull in `subprocess.h`/`posix_spawn_*`, so they share `server-models.cpp`'s Android guard): b9870 exposes `server.cpp`'s entry as `int llama_server(int, char**)` (no `main` in the file), and `patches/0006` makes it embeddable (no process signal handlers, forwarded-argv parse, out-of-band shutdown). **`server-mcp.cpp` is new in b10154** (upstream MCP-server support): both `server.cpp` (`llama_server`'s `mcp_mgr` lifecycle) and `server-tools.cpp` (`tools.setup(..., mcp_mgr)` / `server_mcp::call_tool`) reference `server_mcp`, so it **must** be in the `target_sources` list or the link fails with undefined `server_mcp::{start,shutdown,call_tool,list_tools,~server_mcp}` — **latent on Linux** (a shared object tolerates undefined symbols) but a **hard link error on macOS/ld64 and Windows/MSVC**. It is compiled into `jllama` and — on non-Android — into `jllama_test` too, together with `server.cpp`, `server-tools.cpp` and `server-http.cpp` (and the linked `cpp-httplib` target), so `test_native_server_shutdown.cpp` can drive the real `llama_server()`. The `NativeServer` JNI bridge (`src/main/cpp/native_server.cpp`) calls `llama_server` on a worker thread, so the **full** upstream server — WebUI and all — runs inside `libjllama`. See "Two server modes" below. ### Two server modes (`OpenAiCompatServer` vs `NativeServer`) diff --git a/README.md b/README.md index e4b554593..551c3e6ac 100644 --- a/README.md +++ b/README.md @@ -123,6 +123,7 @@ Inference of Meta's LLaMA model (and others) in pure C/C++. - **Distributed inference over RPC** — offload a model's layers to llama.cpp RPC servers on other machines (`ModelParameters.setRpcServers(...)` / `--rpc host:port`), and serve this machine's devices to them with `RpcServer` (the in-JVM `rpc-server`). See [Distributed inference over RPC](#distributed-inference-over-rpc). - **Local agent** ([`llama-atmosphere-agent`](llama-atmosphere-agent/), release asset, JDK 21+) — a fully offline agent on top of this library that reads and edits files and, if allowed, runs commands. One agent session, three ways to use it: a **terminal** (full console or line-oriented `--plain`, fine over SSH/PuTTY), a **browser** (`--web`, token-protected, loopback by default — reach it from elsewhere through an SSH tunnel), and **IDEs** over the [Agent Client Protocol](https://agentclientprotocol.com) (`--acp`: JetBrains IDEs and Zed natively, VS Code through an ACP extension). See [Local agent](#local-agent-terminal-browser-ide-via-acp). - **Multi-model router mode** (`--models-dir` + per-request model selection, managed via the typed `RouterClient`) and **attach mode** (`NativeServer(LlamaModel, ...)` serves an already-loaded model over the full upstream HTTP frontend — one copy of the weights). +- **HTTPS built in** — model downloads from `https://` URLs (`--model-url`, `-hf`) and a TLS-capable embedded server (`--ssl-key-file`/`--ssl-cert-file`) on every desktop platform, with BoringSSL linked statically: no OpenSSL to install, and no dependency on one (Android and s390x ship without SSL). - Pre-built native binaries for Linux (x86-64, aarch64, s390x), macOS (arm64, Metal included), Windows (x86-64, x86, arm64) and Android (arm64, x86-64), plus GPU backends (CUDA, Vulkan, OpenCL, ROCm/HIP, SYCL, OpenVINO) — one natives jar each, all loadable side by side with automatic CPU fallback; see [Choosing the natives jars](#choosing-the-natives-jars). Android additionally ships as the [`llama-android` AAR](#importing-in-android) with the optional `llama-kotlin` coroutines façade. ## Quick Start @@ -981,8 +982,12 @@ Differences from `OpenAiCompatServer`: with the classic constructor it **loads i the arguments (an independent lifecycle, like `llama-server.exe`), it is **single-instance per process**, it serves the **WebUI** (in released jars — local `cmake` builds ship the empty-asset stub, so no UI there), and it is **not available on Android** (the upstream server needs -`posix_spawn`). Readiness: poll `GET /health`. No SSL (plain HTTP — bind localhost or front with a -TLS proxy). +`posix_spawn`). Readiness: poll `GET /health`. TLS: pass `--ssl-key-file` and `--ssl-cert-file` +(BoringSSL is linked statically into every desktop natives jar, nothing to install). The same +build fetches models from `https://` URLs (`--model-url`, `-hf`), verified against the OS +certificate store -- on Linux `/etc/ssl/certs` or `/etc/ssl/cert.pem`, overridable with +`SSL_CERT_FILE` / `SSL_CERT_DIR`. Note that `--model-url` alone starts the router; name the +download target with `-m` as well. #### Attach mode — serve an already-loaded `LlamaModel` diff --git a/TODO.md b/TODO.md index a2f4b6b07..ffa82a1c3 100644 --- a/TODO.md +++ b/TODO.md @@ -183,18 +183,6 @@ binaries; 2026-10-08) established: look for CUDA hits in the stats table) and flip the default back when a warm run shows them -- the win was ~51 -> ~15 min with CUDA 13.2. -### macOS dylib links Homebrew OpenSSL (found by `verify-native-deps.py`) - -- **The shipped `Mac/aarch64/metal/libjllama.dylib` needs `/opt/homebrew/opt/openssl@3/lib/libssl.3.dylib` - and `libcrypto.3.dylib`** (verified on the published 5.1.0 jar and the current snapshot). The macOS - build finds the runner's Homebrew OpenSSL and links it dynamically, so the macOS natives do not load - on a Mac without `brew install openssl@3` — and the macOS smoke cannot see it, because the runner - has it. Likely fix: build BoringSSL statically on macOS as on Windows - (`LLAMA_BUILD_BORINGSSL`, `llama/CMakeLists.txt`), or turn HTTPS off there (`-DLLAMA_OPENSSL=OFF`; - the library only needs it for URL model downloads). Then delete the two allowlist lines marked - KNOWN DEFECT in `.github/buildcheck/nativedeps.py`. Needs a macOS CI run to verify, which is why it is - not folded into the RPC PR that surfaced it. - ### RPC backend — follow-ups - **A server lost mid-inference still aborts the JVM.** Patch `0015` makes *registration* fail diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index 81f269bf8..2cdf9b489 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -85,7 +85,6 @@ if(ANDROID_ABI) set(LLAMA_BUILD_TOOLS OFF CACHE BOOL "" FORCE) set(LLAMA_BUILD_SERVER OFF CACHE BOOL "" FORCE) endif() -set(LLAMA_CURL OFF) option(LLAMA_VERBOSE "llama: verbose output" OFF) @@ -176,10 +175,24 @@ if(ANDROID OR ANDROID_ABI OR OS_NAME MATCHES "Android" OR CMAKE_CXX_COMPILER MAT endif() set(LLAMA_BUILD_COMMON ON) -# Enable HTTPS model downloads via curl -set(LLAMA_CURL ON) -# Build BoringSSL to include OpenSSL DLLs in Windows packages for HTTPS support -if(WIN32) + +# HTTPS -- model downloads from https:// URLs (common/download.cpp: --model-url, --hf-repo) and the +# embedded server's --ssl-key-file / --ssl-cert-file -- comes from BoringSSL, which upstream's +# vendor/cpp-httplib/CMakeLists.txt fetches at a pinned tag and links STATICALLY, on every platform +# that ships a desktop jar. A shipped library must not depend on an OpenSSL the target machine may +# not have: the macOS build used to pick up the runner's Homebrew openssl@3 and failed to load on a +# Mac without it, and the manylinux images have no OpenSSL that upstream's version check accepts, so +# the Linux jars had no HTTPS at all. Upstream's own release builds pass the same flag (macos-cpu, +# windows-*, android-*). jllama links upstream's cpp-httplib target (further down) instead of +# compiling httplib.cpp a second time, so the download code and the embedded server see ONE copy of +# cpp-httplib built with ONE setting. Off where BoringSSL is not built here: Android (the dockcross +# cross-clang, and the AAR has no download use case) and s390x (big-endian is outside BoringSSL's +# supported set). LLAMA_OPENSSL=OFF there keeps find_package from linking a system OpenSSL by +# accident, which verify-native-deps.py would reject. +if(ANDROID_ABI OR OS_NAME MATCHES "Android" OR OS_ARCH STREQUAL "s390x" OR CMAKE_SYSTEM_PROCESSOR MATCHES "s390x") + set(LLAMA_OPENSSL OFF CACHE BOOL "" FORCE) + set(LLAMA_BUILD_BORINGSSL OFF CACHE BOOL "" FORCE) +else() set(LLAMA_BUILD_BORINGSSL ON CACHE BOOL "" FORCE) endif() @@ -573,22 +586,16 @@ endif() # same 3.12.0 as our FetchContent copy, so nothing added here shadows it. target_sources(jllama PRIVATE ${llama.cpp_SOURCE_DIR}/tools/server/server-http.cpp - ${llama.cpp_SOURCE_DIR}/vendor/cpp-httplib/httplib.cpp ) -# cpp-httplib is third-party: silence its warnings (matching upstream's own -# cpp-httplib target, which compiles it with -w / /w). No SSL is enabled — -# CPPHTTPLIB_OPENSSL_SUPPORT is left undefined — so the embedded server is -# plain-HTTP for now (bind to localhost or front it with a TLS proxy). -if(MSVC) - set_source_files_properties( - ${llama.cpp_SOURCE_DIR}/vendor/cpp-httplib/httplib.cpp - PROPERTIES COMPILE_FLAGS "/w") -else() - set_source_files_properties( - ${llama.cpp_SOURCE_DIR}/vendor/cpp-httplib/httplib.cpp - PROPERTIES COMPILE_FLAGS "-w") -endif() +# cpp-httplib itself is upstream's `cpp-httplib` static target (vendor/cpp-httplib/CMakeLists.txt, +# built whenever LLAMA_BUILD_COMMON is on), linked below -- NOT compiled into jllama a second time. +# That target carries CPPHTTPLIB_OPENSSL_SUPPORT as a PUBLIC definition whenever an SSL library is +# configured (BoringSSL here, see the HTTPS block above), so server-http.cpp, jllama's own TUs and +# common/download.cpp all agree on it. Until 5.2.0 this file compiled httplib.cpp itself, without +# SSL, which put two incompatible copies of the same classes into one link; which copy the linker +# chose decided per platform whether download.cpp got an SSL-capable client (macOS: yes, via the +# Homebrew libssl it then depended on) or not (Linux: an https:// download threw). # MinGW needs ws2_32 explicitly; MSVC auto-links it via a #pragma in httplib.h. if(WIN32 AND NOT MSVC) @@ -622,13 +629,13 @@ target_include_directories(jllama PRIVATE ${llama.cpp_SOURCE_DIR}/tools/server) # Note: the WebUI ui.h include dir (generated webui-generated/ or the empty stub # src/main/cpp/webui_stub) is added by the "WebUI assets" conditional above. -target_link_libraries(jllama PRIVATE llama-common mtmd llama nlohmann_json) +target_link_libraries(jllama PRIVATE llama-common mtmd llama nlohmann_json cpp-httplib) target_compile_features(jllama PRIVATE cxx_std_11) target_compile_definitions(jllama PRIVATE SERVER_VERBOSE=$ - # cpp-httplib tuning — mirror the defines upstream's cpp-httplib target sets so - # httplib.cpp and every TU that includes httplib.h (server-http.cpp) agree on + # cpp-httplib tuning — mirror the defines upstream's cpp-httplib target sets PRIVATELY for + # httplib.cpp, so every TU here that includes httplib.h (server-http.cpp) agrees with it on # the inline behaviour these macros control. CPPHTTPLIB_FORM_URL_ENCODED_PAYLOAD_MAX_LENGTH=1048576 CPPHTTPLIB_LISTEN_BACKLOG=512 @@ -827,12 +834,13 @@ if(BUILD_TESTING) # The generated Java-CLI-flag list consumed by test_model_flags.cpp. ${CMAKE_CURRENT_BINARY_DIR}/generated ) - target_link_libraries(jllama_test PRIVATE llama-common mtmd llama nlohmann_json GTest::gtest_main) + target_link_libraries(jllama_test PRIVATE llama-common mtmd llama nlohmann_json cpp-httplib GTest::gtest_main) target_compile_features(jllama_test PRIVATE cxx_std_17) # The embedded llama-server's shutdown path (test_native_server_shutdown.cpp) drives the real # llama_server() in router mode, so the test binary needs server.cpp and what it pulls in: the - # HTTP transport, its cpp-httplib backend, and the tools/MCP units server.cpp references. Same + # HTTP transport (cpp-httplib itself is the linked upstream target) and the tools/MCP units + # server.cpp references. Same # non-Android guard as jllama (subprocess.h / posix_spawn_*). The empty-asset WebUI stub is # always enough here: no test serves the UI. if(NOT ANDROID_ABI AND NOT OS_NAME MATCHES "Android") @@ -842,7 +850,6 @@ if(BUILD_TESTING) ${llama.cpp_SOURCE_DIR}/tools/server/server-tools.cpp ${llama.cpp_SOURCE_DIR}/tools/server/server-mcp.cpp ${llama.cpp_SOURCE_DIR}/tools/server/server-http.cpp - ${llama.cpp_SOURCE_DIR}/vendor/cpp-httplib/httplib.cpp ) target_include_directories(jllama_test PRIVATE ${CMAKE_SOURCE_DIR}/src/main/cpp/webui_stub) if(MSVC) From 79c6da4ae43920f4d22ff2f553990a828822bfe1 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 9 Oct 2026 20:14:07 +0000 Subject: [PATCH 2/4] Pin the three dockcross images by digest The manifest digests Docker Hub serves for tag 20260712-79e54f9 (the image the wrappers already used), like the manylinux aarch64 image; the wrappers stay dockcross's generated output except for that one line. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2 --- .github/dockcross/dockcross-android-arm64 | 4 +++- .github/dockcross/dockcross-android-x86_64 | 4 +++- .github/dockcross/dockcross-manylinux_2_28-x64 | 4 +++- CHANGELOG.md | 2 ++ REUSE.toml | 5 +++-- 5 files changed, 14 insertions(+), 5 deletions(-) diff --git a/.github/dockcross/dockcross-android-arm64 b/.github/dockcross/dockcross-android-arm64 index 2fe099480..b1b4e53a6 100755 --- a/.github/dockcross/dockcross-android-arm64 +++ b/.github/dockcross/dockcross-android-arm64 @@ -1,6 +1,8 @@ #!/usr/bin/env bash -DEFAULT_DOCKCROSS_IMAGE=dockcross/android-arm64:20260712-79e54f9 +# Pinned by digest (tag 20260712-79e54f9, the manifest Docker Hub served for it on 2026-10-09); +# the one line of this generated wrapper that is edited. Bump it deliberately, with a Publish run. +DEFAULT_DOCKCROSS_IMAGE=dockcross/android-arm64@sha256:60f04acad0d16fccbefc221d0c3419163f5c65f77c5246cb5302380db0762dac #------------------------------------------------------------------------------ # Helpers diff --git a/.github/dockcross/dockcross-android-x86_64 b/.github/dockcross/dockcross-android-x86_64 index dca0778c2..4fa3905ab 100755 --- a/.github/dockcross/dockcross-android-x86_64 +++ b/.github/dockcross/dockcross-android-x86_64 @@ -1,6 +1,8 @@ #!/usr/bin/env bash -DEFAULT_DOCKCROSS_IMAGE=dockcross/android-x86_64:20260712-79e54f9 +# Pinned by digest (tag 20260712-79e54f9, the manifest Docker Hub served for it on 2026-10-09); +# the one line of this generated wrapper that is edited. Bump it deliberately, with a Publish run. +DEFAULT_DOCKCROSS_IMAGE=dockcross/android-x86_64@sha256:d12c4091e9983d4ee5ebb4dad4b6f51420b2e9ec3bf6522d3e32ed87f54ea783 #------------------------------------------------------------------------------ # Helpers diff --git a/.github/dockcross/dockcross-manylinux_2_28-x64 b/.github/dockcross/dockcross-manylinux_2_28-x64 index 163f7bfef..be7c414cf 100755 --- a/.github/dockcross/dockcross-manylinux_2_28-x64 +++ b/.github/dockcross/dockcross-manylinux_2_28-x64 @@ -1,6 +1,8 @@ #!/usr/bin/env bash -DEFAULT_DOCKCROSS_IMAGE=dockcross/manylinux_2_28-x64:20260712-79e54f9 +# Pinned by digest (tag 20260712-79e54f9, the manifest Docker Hub served for it on 2026-10-09); +# the one line of this generated wrapper that is edited. Bump it deliberately, with a Publish run. +DEFAULT_DOCKCROSS_IMAGE=dockcross/manylinux_2_28-x64@sha256:44fa969013dd9a27d25b2f2d2d86651bf961c02c66bce785890a0ea4ff4190fa #------------------------------------------------------------------------------ # Helpers diff --git a/CHANGELOG.md b/CHANGELOG.md index 15506f754..67dadf3b5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -69,6 +69,8 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by in `build.sh` opts back in). With CUDA 13.4 and sccache 0.18.0 the nvcc launcher failed on the first `.cu` files in two confirmed runs (`Missing "cubin" file output`, `Compiler killed by signal 126`), and the uncached retry that followed made those jobs slower than a cold build; the C/C++ objects stay cached. +- **CI: the three dockcross images are pinned by digest** (`android-arm64`, `android-x86_64`, + `manylinux_2_28-x64`, all tag `20260712-79e54f9`), as the manylinux aarch64 image already was. - **Upgraded the pinned llama.cpp from b11529 to b11534**, in 3 reviewed steps, each ending at a tag. No carried patch needed a refresh, and every drop-check still finds its defect. The one step with project code is b11531, **#30210 "chat : refactor API"**: the prompt and the parser state of a chat generation now diff --git a/REUSE.toml b/REUSE.toml index da877bacb..44fcba61b 100644 --- a/REUSE.toml +++ b/REUSE.toml @@ -55,8 +55,9 @@ SPDX-FileCopyrightText = [ SPDX-License-Identifier = "MIT" # The dockcross wrappers are generated, unmodified output of dockcross's own wrapper template -# (imagefiles/dockcross.sh), printed by `docker run --rm dockcross/:`; copyright and -# license as in https://github.com/dockcross/dockcross/blob/master/LICENSE. +# (imagefiles/dockcross.sh), printed by `docker run --rm dockcross/:` -- except for the +# DEFAULT_DOCKCROSS_IMAGE line, which pins the image by digest; copyright and license as in +# https://github.com/dockcross/dockcross/blob/master/LICENSE. [[annotations]] path = [ ".github/dockcross/dockcross-android-arm64", From de6f10d628678573eaf77968bc19d53e50dc22e0 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 9 Oct 2026 20:14:07 +0000 Subject: [PATCH 3/4] Handover: no Mac and no Android device, macOS and Android are CI-only Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2 --- docs/handover/local-agent-pr-a.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/handover/local-agent-pr-a.md b/docs/handover/local-agent-pr-a.md index 4a905cbd5..7c3d5d3bf 100644 --- a/docs/handover/local-agent-pr-a.md +++ b/docs/handover/local-agent-pr-a.md @@ -28,8 +28,8 @@ Everything below is what only your machine can show. 1. **Toolchain inventory** (once): clang versions and paths (LLVM on `PATH`, the one inside Visual Studio under `VC\Tools\Llvm\x64`), Visual Studio and MSVC toolset versions, Windows SDK, Ninja, sccache, JDK(s), Docker images you have pulled, CUDA toolkit and driver version, Vulkan SDK, - Android SDK/NDK. Two questions: do you have access to a Mac (package B1 needs one, otherwise it is - CI-only)? Do you have an Android device, ideally with a Qualcomm Adreno GPU (package E2)? + Android SDK/NDK. (There is no Mac and no Android device; macOS and Android are verified through + CI runners and the CI emulator only, so do not plan work that needs either.) 2. **A1 -- the Java suite with models on Windows.** Build the Ninja CPU library at b11534 the way CI does (`mvn -q compile` in `llama/`, then `.github\build.bat -G "Ninja Multi-Config" -DOS_NAME=Windows From bc23d184cce70421fc08e209b6e8ac3f2d658cdd Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 9 Oct 2026 20:15:13 +0000 Subject: [PATCH 4/4] Linux SYCL builds: no SSL, as upstream's ubuntu-24-sycl job icx/icpx report the compiler id IntelLLVM, which BoringSSL's CMake treats as neither GCC nor Clang; the jars are build-only in CI, so a BoringSSL problem there could not be told apart from a real one. Windows SYCL keeps BoringSSL, as it had before. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2 --- CHANGELOG.md | 3 ++- CLAUDE.md | 11 +++++++---- llama/CMakeLists.txt | 12 ++++++++---- 3 files changed, 17 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 67dadf3b5..5bf5b0d80 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -291,7 +291,8 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by and failed to load on a Mac without that formula (`verify-native-deps.py` carried the two paths as a known defect). Now `jllama` links upstream's `cpp-httplib` target, and BoringSSL is built from source and linked statically on Linux x86-64 / aarch64, macOS and Windows (where it already was), as - upstream's own release builds do; Android and s390x stay without SSL (`LLAMA_OPENSSL=OFF`). The + upstream's own release builds do; Android, s390x and the Linux SYCL jars stay without SSL + (`LLAMA_OPENSSL=OFF`, as upstream's `ubuntu-24-sycl` job). The embedded server's `--ssl-key-file` / `--ssl-cert-file` work as a result. Certificates are verified against the OS store: crypt32 on Windows (the one new import of `jllama.dll`, allowlisted), Security.framework on macOS, `/etc/ssl/certs` and `/etc/ssl/cert.pem` on Linux (`SSL_CERT_FILE` / diff --git a/CLAUDE.md b/CLAUDE.md index 330eefbe7..e07a3cfc7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1299,10 +1299,13 @@ Every desktop natives jar can fetch a model from an `https://` URL (`--model-url (`--ssl-key-file` / `--ssl-cert-file`). The SSL library is **BoringSSL, built from source and linked statically** -- `LLAMA_BUILD_BORINGSSL ON` in `llama/CMakeLists.txt` (the "HTTPS" block), the flag upstream's own release builds pass; upstream's `vendor/cpp-httplib/CMakeLists.txt` fetches the pinned -tag and links `ssl`/`crypto` into its `cpp-httplib` target. **Android and s390x are the exceptions** -(`LLAMA_OPENSSL=OFF`, so `find_package` cannot link a system OpenSSL by accident): the dockcross -cross-clang is not a configuration BoringSSL is built for here, and big-endian is outside its -supported set. `verify-native-deps.py` is what holds the line: a `libssl` of any kind in a shipped +tag and links `ssl`/`crypto` into its `cpp-httplib` target. **Android, s390x and the Linux SYCL jars +are the exceptions** (`LLAMA_OPENSSL=OFF`, so `find_package` cannot link a system OpenSSL by +accident): the dockcross cross-clang is not a configuration BoringSSL is built for here, big-endian +is outside its supported set, and `icx`/`icpx` (compiler id `IntelLLVM`) is neither GCC nor Clang to +BoringSSL's CMake -- upstream's `ubuntu-24-sycl` release job passes `LLAMA_OPENSSL=OFF` as well, +while its `windows-sycl` job (C by `cl`, C++ by `icx`) builds BoringSSL, as the Windows SYCL job here +did before. `verify-native-deps.py` is what holds the line: a `libssl` of any kind in a shipped library's dependency list fails `package`. The one import this adds is Windows' `crypt32.dll` (cpp-httplib's certificate-store lookup), an OS component since Windows 2000, in `WINDOWS_OS`. diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index 2cdf9b489..d276247fa 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -186,10 +186,14 @@ set(LLAMA_BUILD_COMMON ON) # windows-*, android-*). jllama links upstream's cpp-httplib target (further down) instead of # compiling httplib.cpp a second time, so the download code and the embedded server see ONE copy of # cpp-httplib built with ONE setting. Off where BoringSSL is not built here: Android (the dockcross -# cross-clang, and the AAR has no download use case) and s390x (big-endian is outside BoringSSL's -# supported set). LLAMA_OPENSSL=OFF there keeps find_package from linking a system OpenSSL by -# accident, which verify-native-deps.py would reject. -if(ANDROID_ABI OR OS_NAME MATCHES "Android" OR OS_ARCH STREQUAL "s390x" OR CMAKE_SYSTEM_PROCESSOR MATCHES "s390x") +# cross-clang, and the AAR has no download use case), s390x (big-endian is outside BoringSSL's +# supported set) and the Linux SYCL builds (icx/icpx, compiler id IntelLLVM, which BoringSSL's CMake +# treats as neither GCC nor Clang; upstream's ubuntu-24-sycl release job passes LLAMA_OPENSSL=OFF too, +# while its windows-sycl job builds BoringSSL -- as this file did before). LLAMA_OPENSSL=OFF there +# keeps find_package from linking a system OpenSSL by accident, which verify-native-deps.py would +# reject. +if(ANDROID_ABI OR OS_NAME MATCHES "Android" OR OS_ARCH STREQUAL "s390x" OR CMAKE_SYSTEM_PROCESSOR MATCHES "s390x" + OR (GGML_SYCL AND NOT WIN32)) set(LLAMA_OPENSSL OFF CACHE BOOL "" FORCE) set(LLAMA_BUILD_BORINGSSL OFF CACHE BOOL "" FORCE) else()