Skip to content

[CI] Harden ARM64 package access and Docker disk usage#2951

Draft
neverhook wants to merge 3 commits into
kvcache-ai:mainfrom
neverhook:codex/ci-harden-runner-reliability
Draft

[CI] Harden ARM64 package access and Docker disk usage#2951
neverhook wants to merge 3 commits into
kvcache-ai:mainfrom
neverhook:codex/ci-harden-runner-reliability

Conversation

@neverhook

@neverhook neverhook commented Jul 16, 2026

Copy link
Copy Markdown

Summary

  • switch Ubuntu Ports entries from HTTP to HTTPS before the ARM64 CUDA install
  • install job-wide apt IPv4, retry, and timeout policy for all ARM64 package operations
  • make the NVIDIA keyring download use IPv4 with bounded retries
  • free hosted-runner disk before the source Docker build and report disk usage before/after
  • enable plain Docker build progress logs

Motivation

ARM64 package access

The ARM64 matrix failed before Mooncake was configured or built. In PR #2934 run 29479306961, Python 3.10 failed in Install CUDA Toolkit 13.0 (arm64 SBSA) and Python 3.12 was cancelled by matrix fail-fast. PR #2935 reproduced the same failure in run 29479320966. The completed logs show IPv6 is unreachable and all IPv4 connections to ports.ubuntu.com:80 time out while fetching Ubuntu packages.

Docker build capacity

PR #2934 run 29479306961 also failed while linking inside docker/mooncake.Dockerfile with /tmp ... No space left on device; pip then reported [Errno 28]. The same Docker job passed on the larger stacked #2935 SHA, so this is hosted-runner capacity pressure rather than a source error. The cleanup removes large unused hosted-toolcache and Android SDK directories before Buildx starts and leaves disk diagnostics in every run.

This PR is intentionally separate from #2934 and #2935 because both failures are runner/CI reliability issues, not their feature code.

Validation

Local checks:

  • parsed .github/workflows/ci.yml with PyYAML
  • extracted the modified ARM64 and Docker run blocks and passed them through bash -n
  • git diff --check
  • typos-cli 1.30.2 .github/workflows/ci.yml

Remote validation:

After main advanced to 90cdf179, the original commits were rebased without conflict. A blind review then found that step-local apt options did not cover later dependencies.sh package operations; head 9392972f persists the policy through /etc/apt/apt.conf.d, adds the established Android SDK cleanup target, and removes a redundant Docker build argument. Final run 29497850441 is pending on head 9392972f.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Note

Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported.

@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@neverhook
neverhook force-pushed the codex/ci-harden-runner-reliability branch from a06a81d to a10cba8 Compare July 16, 2026 11:24
@neverhook
neverhook force-pushed the codex/ci-harden-runner-reliability branch from 9392972 to 3c73857 Compare July 16, 2026 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants