ci: split CPU builds per-arch on native runners (drop QEMU)#47
Merged
Conversation
Mirror of the GPU split: build amd64 on ubuntu-latest and arm64 on the native ubuntu-24.04-arm runner (no QEMU -- emulation was the bulk of the ML-Spatial build's ~1h runtime), push by digest, then assemble one multi-arch manifest per tag on GHCR and mirror to Docker Hub. Applies to build-ml.yml and build-ml-spatial.yml. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
The cross-registry `imagetools create -t docker.io/... ghcr.io/...` copy fails with 400 Bad Request on Docker Hub blob upload. Push each per-arch image by digest to BOTH GHCR and Docker Hub during the build, then create the multi-arch manifest natively in each registry (no blob copy). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Multiple type=image outputs in one build broke the exported digest (not found in GHCR). Use two single-output push-by-digest steps instead (GHCR, then a cache-hot re-push to Docker Hub) so each exports a correct digest and identical content yields the same digest in both registries. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
The GHCR and Docker Hub pushes produce different digests (provenance / created-timestamp non-determinism), so reusing the GHCR digest against Docker Hub failed (not found). Record each registry's own per-arch digest and assemble each registry's manifest from its own digests. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Mirror of the now-green build-ml.yml: native per-arch builds pushed by digest to both registries, per-registry digests recorded, manifests assembled natively in each (no cross-registry copy). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Mirror of #46 for the CPU images.
build-ml.ymlandbuild-ml-spatial.ymlbuilt amd64+arm64 in one job under QEMU; arm64 emulation was the bulk of the ~1h ML-Spatial runtime. Split each arch onto its own native runner (amd64 → ubuntu-latest, arm64 → ubuntu-24.04-arm), push by digest, merge into one multi-arch manifest per tag on GHCR, mirror to Docker Hub. Keeps arm64 first-class (Apple Silicon, Spark). Needs a CI run to validate (arm64 runner enabled; imagetools mirror).