From cb4b6bef5b952f3287d96fdd96cf8be1d071cb14 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 12 Jul 2026 15:02:34 +0000 Subject: [PATCH] Upgrade llama.cpp from b9972 to b9975 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Internal-only range (CUDA cudaMemGetInfo failure handling, GGUF empty-key rejection, per-sequence DSV4 compressed-cache clearing/isolation fix) — no public API surface touched, all eight local patches re-verified clean. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01WAUg78ZW9bN3U1P6w67GFo --- CLAUDE.md | 8 ++++---- README.md | 2 +- docs/history/llama-cpp-breaking-changes.md | 2 ++ llama/CMakeLists.txt | 4 ++-- .../java/net/ladenthin/llama/value/LlamaCppVersion.java | 8 ++++---- 5 files changed, 13 insertions(+), 11 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 6e182891..e21a06ab 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b9972** +Current llama.cpp pinned version: **b9975** ## Upgrading CUDA Version @@ -421,7 +421,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network; embed.cpp is plain C++17 (no npm) -git clone --depth 1 --branch b9972 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b9975 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build \ && ( cd dist && find . -type f -not -path './_gzip/*' \ | while read -r f; do mkdir -p "_gzip/$(dirname "$f")"; gzip -9 -c "$f" > "_gzip/$f"; done ) \ @@ -461,7 +461,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b9972`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b9975`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -1237,7 +1237,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b9972`. +llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b9975`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.17.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index eb03ba7c..72af61a7 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b9972](https://img.shields.io/badge/llama.cpp-%23b9972-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b9972) +[![llama.cpp b9975](https://img.shields.io/badge/llama.cpp-%23b9975-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b9975) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index 24e66500..d0da4761 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -481,3 +481,5 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b9967–b9968 | upstream verification (sandbox) | All **eight** patches (`0001`–`0008`) re-verified against b9968: applied in filename order onto a clean b9968 checkout via a fresh `cmake -B build` (fail-loud `PATCH_COMMAND`), all clean (the range touches no patch-target file). No OuteTTS generator anchor touched (`tools/tts/tts.cpp` unchanged — the generator extracted `tts.cpp @ b9968`). Full local configure verified (fail-loud patch apply + TTS extraction succeeded); per-platform build + `ctest` confirmation by the CI pipeline. | | b9968–b9972 | `ggml/include/{ggml.h,ggml-rpc.h}` + `ggml/src/{ggml.c,ggml-cpu/**,ggml-vulkan/ggml-vulkan.cpp}` + `src/{llama-context.cpp,llama-cparams.h,llama-graph.{cpp,h},models/deepseek32.cpp,models/deepseek4.cpp}` + `tools/server/{server-context.cpp,server-http.{cpp,h},server-stream.{cpp,h},server-tools.{cpp,h}}` | **Additive-only ggml API, internal-only server refactor, no break** (19 non-test files, ~62 KiB — a 4-commit final chunk). **(1)** New op `GGML_OP_LIGHTNING_INDEXER` + `ggml_lightning_indexer(...)` for DeepSeek Sparse Attention's "lightning indexer" (purely additive; `GGML_OP_COUNT` 97→98, `ggml-rpc.h`'s matching `static_assert`/`RPC_PROTO_PATCH_VERSION` bumped in lockstep — RPC backend not built here); CPU (`ggml-cpu.c`/`ops.{cpp,h}`) and Vulkan implement it, `deepseek32.cpp`/`deepseek4.cpp` wire it into the DeepSeek graph builders; `llama-cparams.h` gains two internal bools (`fused_lid`/`auto_flid`) alongside the existing fused-gated-delta-net flags. **(2)** `tools/server/server-stream.h`'s producer-pipe API is refactored: `stream_pipe_producer::create` now returns a raw pointer (was `shared_ptr`) and the free functions `server_stream_session_attach_pipe`/`server_stream_aware_should_stop` are folded into a new `server_res_spipe` base class (`set_req`/`should_stop`/`on_complete`/`set_next` methods) — `server_res_generator` (`server-context.cpp`) now extends `server_res_spipe` instead of `server_http_res` directly and calls those methods instead of the removed free functions. **(3)** `server-tools.{cpp,h}`'s `server_tool::invoke()` gains a `stream*` parameter (streamed shell-command output) and `tools_io::run()` gains an `on_chunk` callback. None of (2)/(3) touch a patch-target region (patches `0002`/`0005` anchor `server-context.cpp` near `load_model`/`update_slots`, both far from the SSE-response-generator code this chunk touches; `server-stream.h`/`server-tools.{cpp,h}` are not patch targets at all) and no priority-8 header changed. No project source changes required. | | b9968–b9972 | upstream verification (sandbox) | All **eight** patches (`0001`–`0008`) re-verified against b9972: applied in filename order onto a clean b9972 checkout via a fresh `cmake -B build` (fail-loud `PATCH_COMMAND`), all clean. No OuteTTS generator anchor touched (`tools/tts/tts.cpp` unchanged — the generator extracted `tts.cpp @ b9972`). **Full local verification given the larger surface of this chunk:** fresh configure + full `cmake --build` (`jllama` + `jllama_test` link cleanly, confirming the `server_res_spipe` refactor and the new `GGML_OP_LIGHTNING_INDEXER` op compile against this project's TUs) + `ctest` **485/485 passing**; per-platform confirmation by the CI pipeline. | +| b9972–b9975 | `ggml/src/ggml-cuda/ggml-cuda.cu` + `ggml/src/gguf.cpp` + `src/llama-kv-cache-dsv4.{cpp,h}` | **Internal-only, no public-API surface** (4 non-test files, ~15 KiB, 3 commits). `ggml-cuda.cu`'s `ggml_backend_cuda_device_get_memory` now handles a failing `cudaMemGetInfo` gracefully (logs + returns `free=total=0`) instead of aborting via `CUDA_CHECK`. `gguf.cpp` rejects an empty GGUF key during parsing (new `HANDCRAFTED_KV_EMPTY_KEY` fuzz case in `tests/test-gguf.cpp`). `llama-kv-cache-dsv4.{cpp,h}` reworks DSV4 compressed-cache clearing to be **per-sequence** instead of whole-cache: `llama_dsv4_comp_state::clear(bool)` → `clear(llama_seq_id, bool)` and `llama_kv_cache_dsv4::clear_compressed(bool)` → `clear_compressed(llama_seq_id, bool)` (a new `-1` sentinel means "all sequences", matching the old behavior); `seq_cp`/`seq_add`/`seq_div` drop their now-unnecessary full-cache clear and `seq_keep` clears every other sequence individually — an isolation fix so removing one sequence's KV state no longer disturbs sibling sequences (new `test_seq_rm_isolated` regression case in `tests/test-save-load-state.cpp`). Both DSV4 functions are `private` members with no `llama.h`/`common.h` surface; not called by project code. No priority-8 header touched, `tools/ui` untouched, no project source changes required. | +| b9972–b9975 | upstream verification (sandbox) | All **eight** patches (`0001`–`0008`) re-verified against b9975: applied in filename order onto a clean b9975 checkout via a fresh `cmake -B build` (fail-loud `PATCH_COMMAND`), all clean (the range touches no patch-target file). No OuteTTS generator anchor touched (`tools/tts/tts.cpp` unchanged — the generator extracted `tts.cpp @ b9975`). Full local configure verified (fail-loud patch apply + TTS extraction succeeded); full `cmake --build` + `ctest` verification in progress locally, per-platform confirmation by the CI pipeline. | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index f799380f..084c211e 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -173,7 +173,7 @@ set(LLAMA_BUILD_APP OFF CACHE BOOL "" FORCE) FetchContent_Declare( llama.cpp GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git - GIT_TAG b9972 + GIT_TAG b9975 PATCH_COMMAND ${CMAKE_COMMAND} -DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches -DLLAMA_SRC= @@ -196,7 +196,7 @@ execute_process( COMMAND ${CMAKE_COMMAND} -DTTS_SRC=${llama.cpp_SOURCE_DIR}/tools/tts/tts.cpp -DOUT_CPP=${JLLAMA_TTS_GEN_CPP} - -DLLAMA_TAG=b9972 + -DLLAMA_TAG=b9975 -P ${CMAKE_CURRENT_SOURCE_DIR}/cmake/generate-tts-upstream.cmake RESULT_VARIABLE JLLAMA_TTS_GEN_RESULT ) diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index 2b85c560..8edeaed2 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -10,13 +10,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b9972"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b9975"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b9972-0badc06ab"} — call + * plus the resolved upstream commit, e.g. {@code "b9975-0badc06ab"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -24,14 +24,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b9972"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b9975"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b9972"; + public static final String LLAMA_CPP_VERSION = "b9975"; // Constants holder — not instantiable. private LlamaCppVersion() {}