diff --git a/CLAUDE.md b/CLAUDE.md index 5a31fe19..711ee1e7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b9982** +Current llama.cpp pinned version: **b9986** ## Upgrading CUDA Version @@ -421,7 +421,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network; embed.cpp is plain C++17 (no npm) -git clone --depth 1 --branch b9982 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b9986 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build \ && ( cd dist && find . -type f -not -path './_gzip/*' \ | while read -r f; do mkdir -p "_gzip/$(dirname "$f")"; gzip -9 -c "$f" > "_gzip/$f"; done ) \ @@ -461,7 +461,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b9982`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b9986`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -1254,7 +1254,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b9982`. +llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b9986`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.17.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index 8fa9ddaf..d6b189a4 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b9982](https://img.shields.io/badge/llama.cpp-%23b9982-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b9982) +[![llama.cpp b9986](https://img.shields.io/badge/llama.cpp-%23b9986-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b9986) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index b4c897fd..3f510ddc 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -483,5 +483,11 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b9968–b9972 | upstream verification (sandbox) | All **eight** patches (`0001`–`0008`) re-verified against b9972: applied in filename order onto a clean b9972 checkout via a fresh `cmake -B build` (fail-loud `PATCH_COMMAND`), all clean. No OuteTTS generator anchor touched (`tools/tts/tts.cpp` unchanged — the generator extracted `tts.cpp @ b9972`). **Full local verification given the larger surface of this chunk:** fresh configure + full `cmake --build` (`jllama` + `jllama_test` link cleanly, confirming the `server_res_spipe` refactor and the new `GGML_OP_LIGHTNING_INDEXER` op compile against this project's TUs) + `ctest` **485/485 passing**; per-platform confirmation by the CI pipeline. | | b9972–b9975 | `ggml/src/ggml-cuda/ggml-cuda.cu` + `ggml/src/gguf.cpp` + `src/llama-kv-cache-dsv4.{cpp,h}` | **Internal-only, no public-API surface** (4 non-test files, ~15 KiB, 3 commits). `ggml-cuda.cu`'s `ggml_backend_cuda_device_get_memory` now handles a failing `cudaMemGetInfo` gracefully (logs + returns `free=total=0`) instead of aborting via `CUDA_CHECK`. `gguf.cpp` rejects an empty GGUF key during parsing (new `HANDCRAFTED_KV_EMPTY_KEY` fuzz case in `tests/test-gguf.cpp`). `llama-kv-cache-dsv4.{cpp,h}` reworks DSV4 compressed-cache clearing to be **per-sequence** instead of whole-cache: `llama_dsv4_comp_state::clear(bool)` → `clear(llama_seq_id, bool)` and `llama_kv_cache_dsv4::clear_compressed(bool)` → `clear_compressed(llama_seq_id, bool)` (a new `-1` sentinel means "all sequences", matching the old behavior); `seq_cp`/`seq_add`/`seq_div` drop their now-unnecessary full-cache clear and `seq_keep` clears every other sequence individually — an isolation fix so removing one sequence's KV state no longer disturbs sibling sequences (new `test_seq_rm_isolated` regression case in `tests/test-save-load-state.cpp`). Both DSV4 functions are `private` members with no `llama.h`/`common.h` surface; not called by project code. No priority-8 header touched, `tools/ui` untouched, no project source changes required. | | b9972–b9975 | upstream verification (sandbox) | All **eight** patches (`0001`–`0008`) re-verified against b9975: applied in filename order onto a clean b9975 checkout via a fresh `cmake -B build` (fail-loud `PATCH_COMMAND`), all clean (the range touches no patch-target file). No OuteTTS generator anchor touched (`tools/tts/tts.cpp` unchanged — the generator extracted `tts.cpp @ b9975`). Full local configure verified (fail-loud patch apply + TTS extraction succeeded); full `cmake --build` + `ctest` verification in progress locally, per-platform confirmation by the CI pipeline. | -| b9975–b9981 | `common/common.h` + `tools/mtmd/{mtmd.h,mtmd.cpp,mtmd-cli.cpp}` + `tools/server/{server-chat.cpp,server-common.cpp,server-context.cpp,server-models.cpp}` + `vendor/cpp-httplib/{httplib.h,httplib.cpp}` + `scripts/sync_vendor.py` | **One patch-target conflict, otherwise additive (13 files, ~20 KiB, 6 commits).** `common/common.h`'s `common_prompt_checkpoint` gains an `id_task` field (additive). `mtmd.h`'s `mtmd_input_text` gains a `text_len` field alongside `text` so the tokenizer can carry embedded NUL bytes (`mtmd.cpp`/`mtmd-cli.cpp` updated to match — internal to upstream's own callers, not constructed by project code). `server-chat.cpp`'s Anthropic→OAI tool-result conversion now emits multimodal `content_parts` for `image` blocks (previously text-only) — internal to the Anthropic Messages translation path, not a patch target. `server-common.cpp`'s `process_mtmd_prompt` follows the `mtmd_input_text` field split. `server-models.cpp`'s router capability probe adds `LLAMA_ARG_MMPROJ_AUTO` to the offline-safe allowlist and checks `params.no_mmproj`. `vendor/cpp-httplib` bumped 0.49.0→0.50.1 (additive `Get(path, Params, DownloadProgress)` overloads). **The one patch-target hit:** `tools/server/server-context.cpp`'s `update_slots`/`create_checkpoint` picked up upstream's own fix for the exact problem `0005-server-recurrent-near-prompt-end-checkpoints.patch` addressed — `create_checkpoint` now tracks `id_task` and evicts a checkpoint that sits within `checkpoint_min_step` of a newer one, and `do_checkpoint`'s min-step exemption now covers **every** near-prompt-end checkpoint (`near_prompt_end`, unconditional) rather than only recurrent/hybrid models as `0005` scoped it — a strict superset of `0005`'s behavior. `0005` no longer applies (`patch does not apply` at `server-context.cpp:3560`) and was **dropped** rather than refreshed; see the CLAUDE.md patches-table note under `0005`'s old slot. No other patch-target file (`common/arg.*`, `server.cpp`, `server-models.cpp`'s worker-cmd region) touched; all eight priority-8 headers otherwise byte-identical (only `mtmd.h`, additive). | +| b9975–b9981 | `common/common.h` + `tools/mtmd/{mtmd.h,mtmd.cpp,mtmd-cli.cpp}` + `tools/server/{server-chat.cpp,server-common.cpp,server-context.cpp,server-models.cpp}` + `vendor/cpp-httplib/{httplib.h,httplib.cpp}` + `scripts/sync_vendor.py` | **One patch-target conflict, otherwise additive (13 files, ~20 KiB, 6 commits).** `common/common.h`'s `common_prompt_checkpoint` gains an `id_task` field (additive). `mtmd.h`'s `mtmd_input_text` gains a `text_len` field alongside `text` so the tokenizer can carry embedded NUL bytes (`mtmd.cpp`/`mtmd-cli.cpp` updated to match — internal to upstream's own callers, not constructed by project code). `server-chat.cpp`'s Anthropic→OAI tool-result conversion now emits multimodal `content_parts` for `image` blocks (previously text-only) — internal to the Anthropic Messages translation path, not a patch target. `server-common.cpp`'s `process_mtmd_prompt` follows the `mtmd_input_text` field split. `server-models.cpp`'s router capability probe adds `LLAMA_ARG_MMPROJ_AUTO` to the offline-safe allowlist and checks `params.no_mmproj`. `vendor/cpp-httplib` bumped 0.49.0→0.50.1 (additive `Get(path, Params, DownloadProgress)` overloads). **The one patch-target hit — landed at the single sub-tag b9977→b9978, not spread across the range:** `tools/server/server-context.cpp`'s `update_slots`/`create_checkpoint` picked up upstream's own fix for the exact problem `0005-server-recurrent-near-prompt-end-checkpoints.patch` addressed — `create_checkpoint` now tracks `id_task` and evicts a checkpoint that sits within `checkpoint_min_step` of a newer one, and `do_checkpoint`'s min-step exemption now covers **every** near-prompt-end checkpoint (`near_prompt_end`, unconditional) rather than only recurrent/hybrid models as `0005` scoped it — a strict superset of `0005`'s behavior. Per-sub-tag isolation (`git diff b b` for each intermediate step, checked against every patch-target file): b9975→b9976 and b9976→b9977 touch no patch-target file; **b9977→b9978 is this commit**; b9978→b9979 (`server-common.cpp`'s `process_mtmd_prompt`, matching the `mtmd_input_text` field split above) and b9979→b9980 (`server-models.cpp`'s `update_caps()`, the `LLAMA_ARG_MMPROJ_AUTO` allowlist entry above) touch patch-target *files* but not patch-target *regions* (`0008` edits `update_args()` in the same file, a different function); b9980→b9981 touches none. `0005` no longer applies (`patch does not apply` at `server-context.cpp:3560`) and was **dropped** rather than refreshed; see the CLAUDE.md patches-table note under `0005`'s old slot. No other patch-target file (`common/arg.*`, `server.cpp`, `server-models.cpp`'s worker-cmd region) touched; all eight priority-8 headers otherwise byte-identical (only `mtmd.h`, additive). | | b9975–b9981 | upstream verification (sandbox) | **Seven** patches (`0001`–`0004`, `0006`–`0008`) re-verified against b9981: applied in filename order onto a clean b9981 checkout via a fresh `cmake -B build` (fail-loud `PATCH_COMMAND`), all clean after removing `0005` (superseded — see the row above). No OuteTTS generator anchor touched (`tools/tts/tts.cpp` unchanged — the generator extracted `tts.cpp @ b9981`). Full local verification: fresh configure (fail-loud patch apply + TTS extraction succeeded, `ggml commit 34558825a`) + full `cmake --build` (`jllama` + `jllama_test` link cleanly) + `ctest` **485/485 passing**; per-platform confirmation by the CI pipeline. | +| b9981–b9982 | `tools/server/server-common.cpp` + `tests/test-chat.cpp` | **Upstream merged an open PR we were carrying locally, no project source changes required (2 files, ~2 KiB, 1 commit).** `oaicompat_chat_params_parse`'s reasoning-budget block now reads `reasoning_budget_tokens` before falling back to the Anthropic `thinking_budget_tokens` alias and then the server default, and threads a per-request `reasoning_budget_message` override through — byte-for-byte the same precedence logic as `0004-pr23116-server-per-request-reasoning-budget-tokens.patch` (which carried [ggml-org/llama.cpp#23116](https://github.com/ggml-org/llama.cpp/pull/23116)). `tests/test-chat.cpp` gains the identical two regression cases (`test_reasoning_budget_tokens_per_request` / `test_reasoning_budget_message_per_request`) our patch also carried. `0004` was **dropped**, not refreshed — the tree already matches what it used to add; see the CLAUDE.md patches-table note under `0004`'s old slot. No patch-target file other than `server-common.cpp` touched (`0001`'s `common/arg.*` block, `server.cpp`, `server-context.cpp`, `server-models.cpp`'s worker-cmd region all untouched in this range); no priority-8 header changed. | +| b9981–b9982 | upstream verification (sandbox) | **Six** patches (`0001`–`0003`, `0006`–`0008`) re-verified against b9982: applied in filename order onto a clean b9982 checkout via a fresh `cmake -B build` (fail-loud `PATCH_COMMAND`), all clean after removing `0004` (merged upstream — see the row above). No OuteTTS generator anchor touched (`tools/tts/tts.cpp` unchanged — the generator extracted `tts.cpp @ b9982`). Full local verification: fresh configure (fail-loud patch apply + TTS extraction succeeded, `ggml commit 99f3dc322`) + full `cmake --build` (`jllama` + `jllama_test` link cleanly) + `ctest` **485/485 passing**; per-platform confirmation by the CI pipeline. | +| b9982–b9984 | none (patch-target-relevant) | **Additive-only, no public-API surface (2 files, ~9 KiB excluding the auto-followed `tools/ui` WebUI rework, 2 commits).** `ggml/src/ggml-sycl/{dmmv.cpp,ggml-sycl.cpp}` gain new dequantize-matrix-vector kernel variants — entirely inside the SYCL backend's upstream-compiled TUs (this project's SYCL classifiers are build-only, no source touched). No `common/arg.*`, `tools/server/*`, or OuteTTS generator anchor (`tools/tts/tts.cpp` unchanged) touched; all eight priority-8 headers byte-identical. All **six** remaining patches (`0001`–`0003`, `0006`–`0008`) apply unchanged — confirmed via the upstream `b9982...b9984` compare diff. | +| b9982–b9984 | upstream verification (sandbox) | All **six** patches (`0001`–`0003`, `0006`–`0008`) re-verified against b9984: applied in filename order onto a clean b9984 checkout via a fresh `cmake -B build` (fail-loud `PATCH_COMMAND`), all clean. No OuteTTS generator anchor touched (`tools/tts/tts.cpp` unchanged — the generator extracted `tts.cpp @ b9984`). Full local verification: fresh configure (fail-loud patch apply + TTS extraction succeeded, `ggml commit e474bba7a`) + full `cmake --build` (`jllama` + `jllama_test` link cleanly) + `ctest` **485/485 passing**; per-platform confirmation by the CI pipeline. | +| b9984–b9986 | `common/{chat-auto-parser-generator.cpp,chat-diff-analyzer.cpp}` + `ggml/src/ggml-sycl/{backend.hpp,common.hpp,ggml-sycl.cpp,topk-moe.{cpp,hpp}}` + `tests/test-chat.cpp` | **Additive/tuning-only, no public-API surface (9 files, ~32 KiB excluding the auto-followed `tools/ui` WebUI rework, 2 commits).** `chat-auto-parser-generator.cpp`/`chat-diff-analyzer.cpp` fix a whitespace-trimming and preserved-tokens bug in the NVIDIA-Nemotron-Nano-v2 auto-parser profile (`reasoning.start` trimmed before use; `tools_array_wrapped` flag added; a stale `` preserved-token entry dropped) — neither file is a patch target or in the priority-8 header list, and this project links `chat.cpp`/`chat.h` (unchanged) rather than these newer split-out analyzer files directly, but they compile into the same upstream-compiled `llama-common` static lib either way. `tests/test-chat.cpp` gains matching regression cases for the fixed template output (not a patch target). `ggml/src/ggml-sycl/*` gains new top-k MoE (mixture-of-experts) kernels — entirely inside the SYCL backend's upstream-compiled TUs (this project's SYCL classifiers are build-only, no source touched). No `common/arg.*`, `tools/server/*`, or OuteTTS generator anchor (`tools/tts/tts.cpp` unchanged) touched; all eight priority-8 headers byte-identical. All **six** remaining patches (`0001`–`0003`, `0006`–`0008`) apply unchanged — confirmed via the upstream `b9984...b9986` compare diff. This chunk reaches the latest release at the time of the bump (b9986). | +| b9984–b9986 | upstream verification (sandbox) | All **six** patches (`0001`–`0003`, `0006`–`0008`) re-verified against b9986: applied in filename order onto a clean b9986 checkout via a fresh `cmake -B build` (fail-loud `PATCH_COMMAND`), all clean. No OuteTTS generator anchor touched (`tools/tts/tts.cpp` unchanged — the generator extracted `tts.cpp @ b9986`). Full local verification: fresh configure (fail-loud patch apply + TTS extraction succeeded, `ggml commit 91c631b21`) + full `cmake --build` (`jllama` + `jllama_test` link cleanly) + `ctest` **485/485 passing**; per-platform confirmation by the CI pipeline. | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index a9a142c6..c1d59bb8 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -173,7 +173,7 @@ set(LLAMA_BUILD_APP OFF CACHE BOOL "" FORCE) FetchContent_Declare( llama.cpp GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git - GIT_TAG b9982 + GIT_TAG b9986 PATCH_COMMAND ${CMAKE_COMMAND} -DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches -DLLAMA_SRC= @@ -196,7 +196,7 @@ execute_process( COMMAND ${CMAKE_COMMAND} -DTTS_SRC=${llama.cpp_SOURCE_DIR}/tools/tts/tts.cpp -DOUT_CPP=${JLLAMA_TTS_GEN_CPP} - -DLLAMA_TAG=b9982 + -DLLAMA_TAG=b9986 -P ${CMAKE_CURRENT_SOURCE_DIR}/cmake/generate-tts-upstream.cmake RESULT_VARIABLE JLLAMA_TTS_GEN_RESULT ) diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index 729726d0..04433348 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -10,13 +10,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b9982"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b9986"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b9982-0badc06ab"} — call + * plus the resolved upstream commit, e.g. {@code "b9986-0badc06ab"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -24,14 +24,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b9982"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b9986"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b9982"; + public static final String LLAMA_CPP_VERSION = "b9986"; // Constants holder — not instantiable. private LlamaCppVersion() {}