build(arm64): cross-compile parakeet-cli for the Raspberry Pi 5#32
Open
richiejp wants to merge 3 commits into
Open
build(arm64): cross-compile parakeet-cli for the Raspberry Pi 5#32richiejp wants to merge 3 commits into
richiejp wants to merge 3 commits into
Conversation
Owner
|
@richiejp can we hook up to the CI images so it gets published? |
Contributor
Author
|
I guess there are two ways to go about this, one is to publish a docker image, which I'm not sure it makes sense for Rpi or do true static compilation including libc and OpenMP. The later makes most sense to me if it is possible. I'll give it a shot |
Static, self-contained aarch64 build via docker buildx + qemu, mirroring the
privacy-filter.cpp approach. Base ubuntu:24.04 matches the Pi 5's glibc 2.39 ABI
so the binary drops straight on; it targets the Cortex-A76
(armv8.2-a+dotprod+fp16) and links only the system libc/libstdc++/libgomp (ggml
and libparakeet are static).
docker buildx build --platform linux/arm64 -f docker/Dockerfile.arm64 \
--target export --output type=local,dest=build/arm64 .
The build context drops .git (.dockerignore), which dangles the ggml submodule's
gitlink and breaks scripts/apply_ggml_patches.sh (it uses `git apply`). Re-init a
throwaway git repo in the submodule tree before configure so the in-tree ggml
patches (the sgemm broadcast-fold, etc.) apply -- worth ~9% at 4 threads on the Pi.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Binary-only AppImages (no model inside) for aarch64 and x86_64. packaging/appimage/make-appimage.sh is the single packaging path: it assembles the AppDir (parakeet-cli plus the only two non-libc dynamic deps, libstdc++/libgomp, resolved from the running system), fetches appimagetool and the type2 runtime, and packs parakeet-cli-<version>-<arch>.AppImage. glibc stays system-provided, so the AppImage carries the same glibc floor as the raw binary. docker/Dockerfile.arm64 becomes docker/Dockerfile.static: same static build, but the arch flags now key off TARGETARCH (arm64 keeps the Pi 5 Cortex-A76 default; amd64 pins a portable Haswell+ AVX2 baseline since GGML_NATIVE=OFF) and a new appimage/export-appimage stage pair packages both arches in one buildx invocation. The downloaded appimagetool gets its three AppImage magic bytes zeroed before use: qemu-user's binfmt mask requires the ELF e_ident pad bytes to be zero, so the aarch64 tool otherwise dies with "Exec format error" under emulation. release.yml grows a Package/Smoke/Upload AppImage step trio on the linux cpu matrix entries (native runners, same script), so tagged releases ship one grab-and-run file per arch next to the tarballs. vulkan/cuda stay tarball-only -- their binaries need system GPU stacks an AppImage can't sensibly bundle. Verified: x86_64 AppImage in a clean ubuntu:24.04 container and the aarch64 one on a Pi 5 (direct FUSE exec) both print --version and transcribe the speech.wav fixture byte-identically to the known-good text. Co-Authored-By: Claude Fable 5 <[email protected]>
Pi 5 Model B Rev 1.1 (8 GB, Active Cooler, Raspberry Pi OS trixie),
static aarch64 build from docker/Dockerfile.static, governor pinned to
performance. scripts/bench_rpi5.sh drives parakeet-cli bench over a
model x threads{1,2,4} x clip{7.4s,67s} matrix with a 2s
temp/clock/fan/throttle sampler beside every run; benchmarks/rpi5.md
has the tables, benchmarks/results/rpi5/ the raw JSON + telemetry.
Headlines: tdt_ctc-110m-q8_0 hits 18x real time at 4 threads;
tdt-0.6b-v3-q8_0 and nemotron q8_0 hold 2.8-4.6x at 4 threads and stay
>= 1x single-threaded on the long clip at ~1.0 GB peak RSS. The fan
answers the question it was bought for: all 24 cells ran the full
2.4 GHz with throttled=0x0, peaking at 77.1 C against the 85 C soft
limit. All four models transcribe the fixture correctly on-device
(nemotron with no LID-tag leakage).
Co-Authored-By: Claude Fable 5 <[email protected]>
richiejp
force-pushed
the
build/arm64-cross-compile
branch
from
385592b to
6104ab9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Static, self-contained aarch64 build via docker buildx + qemu, mirroring the
privacy-filter.cpp approach. Base ubuntu:24.04 matches the Pi 5's glibc 2.39 ABI
so the binary drops straight on; it targets the Cortex-A76
(armv8.2-a+dotprod+fp16) and links only the system libc/libstdc++/libgomp (ggml
and libparakeet are static).
docker buildx build --platform linux/arm64 -f docker/Dockerfile.arm64
--target export --output type=local,dest=build/arm64 .
The build context drops .git (.dockerignore), which dangles the ggml submodule's
gitlink and breaks scripts/apply_ggml_patches.sh (it uses
git apply). Re-init athrowaway git repo in the submodule tree before configure so the in-tree ggml
patches (the sgemm broadcast-fold, etc.) apply -- worth ~9% at 4 threads on the Pi.
Co-Authored-By: Claude Opus 4.8 (1M context) [email protected]