Skip to content

Make the release pipeline testable before a tag (dry-run + auto rehearsal)#82

Merged
Reefact merged 4 commits into
mainfrom
claude/cicd-audit-4crp4d
Jul 10, 2026
Merged

Make the release pipeline testable before a tag (dry-run + auto rehearsal)#82
Reefact merged 4 commits into
mainfrom
claude/cicd-audit-4crp4d

Conversation

@Reefact

@Reefact Reefact commented Jul 10, 2026

Copy link
Copy Markdown
Owner

Summary

release.yml is the only workflow no CI run exercises before a real tag — version resolution, pack, SBOM generation, OIDC and the attestation permissions only ever execute in production, on a tag, once. Several defects were found here that were invisible from the repository (the +metadata regex, the nuget.org re-signing mismatch, the dispatch-tag target). This makes the pipeline testable two complementary ways, split by side effect, with the packing logic single-sourced so the rehearsal cannot drift, and documents the manual path.

  1. a manual dry-run in release.yml that runs everything up to and including the attestation (the near-real rehearsal, on demand);
  2. an automatic dry-run workflow that runs the side-effect-free portion (build + pack + SBOM) on every PR and push to main;
  3. a shared pack script both workflows call, so the dry-run can never diverge from the real release;
  4. maintainer documentation (EN + FR) for the manual dry-run.

Type of change

  • Bug fix
  • New feature
  • Breaking change
  • Refactoring
  • Analyzer / diagnostic change
  • Tests
  • Documentation
  • Build / CI / tooling

Changes

tools/packaging/pack.sh — single source of truth (new)

  • packs the two published projects with GenerateSBOM=true and then asserts the SPDX manifest is embedded in each .nupkg. Both release.yml and release-dryrun.yml call it — the same pattern the repo already uses for commit-lint (one script shared by the hook and CI), so the dry-run cannot drift from the real release, and the real release gains the SBOM-present check too. (Named packaging/ because the .gitignore [Rr]elease/ rule would ignore tools/release/.)

.github/workflows/release.yml — manual dry-run mode

  • new dry_run boolean input on workflow_dispatch, defaulting to true so an accidental dispatch publishes nothing — publishing requires explicitly unticking it;
  • the Pack step now calls tools/packaging/pack.sh;
  • a dry run executes everything up to and including the provenance attestation (deliberately — OIDC/permission failures are exactly the class it rehearses) and skips the three publish steps (NuGet login, push, GitHub Release), gated on github.event_name == 'push' || inputs.dry_run == false, so a tag push always publishes — the normal release path is unchanged.

.github/workflows/release-dryrun.yml — automatic rehearsal (new)

  • runs on every push to main, every PR, and workflow_dispatch; contents: read only — no OIDC, no attestation, none of release.yml's write scopes;
  • builds the solution, then calls the same tools/packaging/pack.sh; deliberately stops before every side-effecting step (no attestation, no NuGet push, no GitHub Release).

maintainers/ — documentation (new)

  • ReleaseDryRun.en.md + ReleaseDryRun.fr.md: pedagogical, bilingual maintainer doc for the manual dry-run (purpose, per-step table, how to run it, impacts — notably that the attestation does run and writes a permanent public record). In a new top-level maintainers/ directory, deliberately separate from doc/ (the library's user documentation).

Testing

  • dotnet build FirstClassErrors.sln
  • dotnet test FirstClassErrors.sln
  • Analyzer tests pass (FirstClassErrors.Analyzers.UnitTests)

The automatic dry-run validates itself on this PR (it triggers on pull_request). The shared script was also rehearsed locally on the .NET 10 SDK under the CI warning ratchet: build green (0 warnings), both projects pack via tools/packaging/pack.sh, and the SBOM-present check passes (exit 0). YAML and POSIX-sh syntax both validate.

The release.yml manual dry-run remains CI-only by nature (it needs OIDC); its first workflow_dispatch run after merge is the proof — e.g. version 0.0.0-dry.1, dry_run ticked. Neither path can exercise the actual NuGet push / repository-signed bytes; only a real release does.

Documentation

  • No library documentation change required (the new maintainers/ doc is operational/maintainer documentation, kept separate from doc/).

🤖 Generated with Claude Code

https://claude.ai/code/session_01AbXi2ZEaUFeFWodNxdY6Zk

@Reefact Reefact changed the title Release: add a dry-run mode to make the workflow testable before a tag Make the release pipeline testable before a tag (dry-run + auto rehearsal) Jul 10, 2026
claude added 3 commits July 10, 2026 17:08
release.yml is the only workflow no CI run exercises before a real tag:
version resolution, pack, SBOM generation, OIDC and the attestation
permissions only ever execute in production conditions, and three defects
invisible from the repository were found here (the +metadata regex, the
nuget.org re-signing mismatch, the dispatch-tag target).

Add a dry_run boolean input to workflow_dispatch (default TRUE, so an
accidental dispatch publishes nothing; publishing requires explicitly
unticking it). A dry run executes everything up to and INCLUDING the
provenance attestation — deliberately, since OIDC/permission failures are
exactly the class of error it rehearses — and skips the three publish steps
(NuGet login, push, GitHub Release), all gated on:

  github.event_name == 'push' || inputs.dry_run == false

A tag push short-circuits on the event check and always publishes, so the
normal release path is unchanged.

Verified: YAML parses; the step matrix shows the nine build/attest steps
unconditional and exactly the three publish steps gated. The dry run itself
is CI-only by nature (OIDC) — its first execution after merge is the proof,
and is precisely what this mode exists to make possible.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01AbXi2ZEaUFeFWodNxdY6Zk
release.yml runs only on a tag or manual dispatch, so its packaging path is
otherwise exercised for the first time in production — on a tag, once. This
adds a read-only workflow that continuously rehearses the pipeline's
side-effect-free portion (build, pack, embed SBOM) for the two published
projects, on every push to main and pull request, catching packaging/SBOM
regressions in ordinary CI.

It stops before every side-effecting step: no provenance attestation (that
writes a permanent public record — kept to release.yml's manual dispatch
dry-run and to real releases), no NuGet login/push (nuget.org has no dry-run
push), no GitHub Release. Permissions are therefore contents: read only, with
no OIDC — narrower than release.yml's write scopes.

The Build/Pack steps mirror release.yml exactly (same projects, --no-build,
GenerateSBOM), and a positive check asserts the SPDX manifest is actually
embedded in each .nupkg — a silent GenerateSBOM regression fails the job
rather than passing green.

Verified locally on the .NET 10 SDK under the CI warning ratchet: build is
green (0 warnings), both packages pack, and each embeds
_manifest/spdx_2.2/manifest.spdx.json. Not covered (by nature): the actual
NuGet push and repository-signed bytes, which only a real release exercises.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01AbXi2ZEaUFeFWodNxdY6Zk
Add a pedagogical, bilingual maintainer/operational doc for the manual dry run
introduced in this branch: what it is, what it does and skips (a per-step
table), how to run it from the Actions tab, and its impacts — notably that the
attestation step DOES run and writes a permanent public record, so a fake
0.0.0-dry.N version should be used and it should be run deliberately, not in a
loop. It also points to the automatic release-dryrun workflow for the
side-effect-free part, and is honest that neither dry run can test the actual
nuget.org push.

Placed under a new top-level maintainers/ directory (ReleaseDryRun.en.md +
ReleaseDryRun.fr.md), deliberately separate from doc/ (the library's user
documentation), following the repo's .en.md / .fr.md bilingual convention.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01AbXi2ZEaUFeFWodNxdY6Zk
@Reefact Reefact force-pushed the claude/cicd-audit-4crp4d branch from 344feec to 5fc1578 Compare July 10, 2026 17:08
release.yml and release-dryrun.yml both packed the two published projects with
their SBOM by copy-pasting the same dotnet pack commands — if one changed (a
third packable project, a flag, the SBOM property), the automatic dry run would
silently stop mirroring the real release, defeating its purpose.

Extract the packing into tools/packaging/pack.sh, the single source of truth
(same pattern the repo already uses for commit-lint, shared between the hook and
CI). It packs the two projects with GenerateSBOM and then asserts the SPDX
manifest is actually embedded in each .nupkg; both workflows now call it, so the
rehearsal cannot drift, and the real release gains the same SBOM-present check.

Named tools/packaging/ rather than tools/release/ because the .gitignore
[Rr]elease/ rule (build output) would otherwise ignore the script.

Verified locally on the .NET 10 SDK: build the solution, then run the shared
script with a throwaway version — both packages are produced and the SBOM check
passes (exit 0). YAML and POSIX-sh syntax both validate.
@Reefact Reefact merged commit 2318abe into main Jul 10, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants