Adding Experimental llama.cpp Hexagon Build#15
Merged
Merged
Conversation
infil00p
commented
Feb 26, 2026
infil00p
commented
Contributor
- Added Hexagon support to Baseweight Snap using the llama.cpp shared libraries generated by the Hexagon Docker
- Fixed a bug with stale context when loading new images
Add multi-backend build system (vulkan/hexagon product flavors) that supports both building Vulkan from source and linking against prebuilt Snapdragon Docker libraries for Hexagon NPU + OpenCL GPU acceleration. Key changes: - CMakeLists.txt: dual backend support with IMPORTED prebuilt libraries for hexagon flavor, building common/cpp-httplib from source - build.gradle.kts: product flavors (vulkan/hexagon) with per-flavor ABI filters and jniLibs packaging - AndroidManifest.xml: uses-native-library for libOpenCL.so and libcdsprpc.so (vendor library namespace access), extractNativeLibs - MTMD_Android.kt: pre-load vendor OpenCL before native library init - mtmd-android.cpp: set ADSP_LIBRARY_PATH for Hexagon DSP skel lookup - Update llama.cpp submodule to 35715657c (matches prebuilt binaries) - Add OpenCL-Headers as submodule for Adreno GPU backend - Add hexagon build documentation and build script Tested on OnePlus 13 (Snapdragon 8 Elite, Hexagon v81): - OpenCL GPU: Adreno 840, all layers offloaded, working - Hexagon NPU: session opens, model loads, but crashes in ggml_hexagon_session::flush() during inference (experimental backend) Co-Authored-By: Claude Opus 4.6 <[email protected]>
The Hexagon backend crashed with SIGABRT in ggml_hexagon_session::flush() during the first real llama_decode call. Two differences from the working llama-mtmd-cli were identified: 1. Missing warmup: the CLI runs a warmup decode (llama_set_warmup + dummy tokens) to let backends compile and validate compute graphs before real data. Without this, Hexagon hit uninitialized state on first decode. 2. swa_full defaulted to true (llama_context_default_params), while the CLI uses false via common_params. This changes KV cache sizing and graph structure in ways the experimental Hexagon backend doesn't handle. Co-Authored-By: Claude Opus 4.6 <[email protected]>
Clear n_past, KV cache, and sampler state at the start of each generation so that taking a new photo starts with a fresh context instead of accumulating stale embeddings from previous images. Co-Authored-By: Claude Opus 4.6 <[email protected]>
The addition of product flavors (vulkan/hexagon) renamed the test task. Hexagon flavor can't build in CI (needs prebuilt Snapdragon libraries). Co-Authored-By: Claude Opus 4.6 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.