ci: R2-backed cache-cargo-home action for cargo download caches#3624
ci: R2-backed cache-cargo-home action for cargo download caches#3624sxlijin wants to merge 2 commits into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4c40bcfb59
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| try { | ||
| if (!(await fileExists(src))) return; // build didn't end up fetching it. | ||
| const key = crateObjectKey(CRATES_IO_INDEX, c.crateFile); | ||
| if (await store.has(key)) return; // another job already uploaded it. |
There was a problem hiding this comment.
When restore marks a crate missed because the cached bytes failed the Cargo.lock checksum, the post step reaches this path after Cargo has downloaded the correct .crate. Since the R2 key is only name-version, store.has(key) is still true for the corrupt object, so this return leaves the bad cache entry in place and every future job will redownload the crate after the same checksum mismatch; keying by checksum or overwriting objects for checksum-mismatch misses would let the cache self-heal.
Useful? React with 👍 / 👎.
|
No description provided. |
Merging this PR will not alter performance
Comparing 
Footnotes
|
sxlijin
force-pushed
the
sam/cache-cargo-home
branch
from
4c40bcf to
8300850
Compare
sxlijin
force-pushed
the
sam/cache-cargo-home
branch
from
8300850 to
0998d85
Compare
sxlijin
force-pushed
the
sam/cache-cargo-home
branch
from
0998d85 to
a7871ab
Compare
sxlijin
force-pushed
the
sam/cache-cargo-home
branch
from
a7871ab to
ef692dc
Compare
sxlijin
force-pushed
the
sam/cache-cargo-home
branch
from
ef692dc to
29377f8
Compare
Route RUSTC_WRAPPER through a baml-sccache wrapper (tools/baml-sccache on POSIX, the native tools_sccache crate on Windows) backed by R2, configured entirely from .envrc so CI matches local shells. mise installs sccache + direnv; each cargo job loads .envrc via `direnv export gha`. Swatinem/rust-cache still caches the cargo registry/git download state, with cache-targets: false so sccache owns target/ and the two caches don't compete. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Add .github/actions/cache-cargo-home, a node20 action that reads baml_language/Cargo.lock and restores ~/.cargo/{registry/cache,registry/index,git/db} from R2 at job start, then uploads anything newly fetched at job end — replacing Swatinem/rust-cache for the download caches (sccache still owns target/).
Granular and Cargo.lock-driven: one object per crate, per sparse-index entry, and per git db, so the cache is shared across jobs and tolerant to Cargo.lock changes. R2 access is hand-rolled SigV4 (no AWS SDK); large git dbs upload via presigned S3 multipart.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
sxlijin
force-pushed
the
sam/cache-cargo-home
branch
from
29377f8 to
5d6de7c
Compare
Use sccache (R2-backed) for Rust **compilation** artifacts in the cargo CI jobs, configured entirely from `.envrc` so CI matches local shells. - `tools_sccache` crate / `tools/baml-sccache` wrapper: a `RUSTC_WRAPPER` that maps `BAML_SCCACHE_R2_*` → `AWS_*` and execs sccache (native crate on Windows, shell script on POSIX). - mise installs sccache + direnv; each cargo job loads `.envrc` via `direnv export gha` (the single source of truth for the sccache/R2 config). - **Swatinem/rust-cache still caches the cargo registry/git download state**, with `cache-targets: false` so sccache owns `target/` and the two caches don't compete. Fork PRs without R2 secrets fall back to the runner-local cache. Follow-up BoundaryML#3624 replaces Swatinem for the download caches with a granular, Cargo.lock-driven R2 action (`cache-cargo-home`); this PR is the sccache base it stacks on. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
1 participant
Stacked on #3612 (sccache). Merge #3612 first.
Adds
.github/actions/cache-cargo-home, a node20 action that readsbaml_language/Cargo.lockand restores~/.cargo/{registry/cache,registry/index,git/db}from R2 at job start, then uploads anything newly fetched at job end — replacing Swatinem/rust-cache for the cargo download caches (sccache still ownstarget/).crates/<index>/<name>-<ver>.crate), per sparse-index entry (registry-index/<index>/<sharded-name>), and per git db (git-db/<ident>-<hash>.tar). The cache is shared across jobs and tolerant to Cargo.lock changes (each lockfile reads the subset it needs).config.jsonlets cargo resolve offline, skipping the online "Updating crates.io index" pass.The action is fully unit-tested (incl. live R2 round-trips) and validated across linux/macOS/Windows.
🤖 Generated with Claude Code