Skip to content

chore: prepare MCPAudit 2.5.0 release candidate - #171

Merged
saagpatel merged 9 commits into
mainfrom
agent/release-2.5.0
Jul 19, 2026
Merged

chore: prepare MCPAudit 2.5.0 release candidate#171
saagpatel merged 9 commits into
mainfrom
agent/release-2.5.0

Conversation

@saagpatel

@saagpatel saagpatel commented Jul 19, 2026

Copy link
Copy Markdown
Owner

Summary

Prepare MCPAudit 2.5.0 as a reviewable, intentionally non-publishable release candidate. The candidate packages Proof Before Action and the security repairs already landed on main, aligns package/server/changelog metadata, and separates tag creation from PyPI publication with an exact tag/commit-bound manual workflow.

Changes

  • Set the candidate version to 2.5.0 across pyproject.toml, server.json, uv.lock, CLI tests, and the changelog.
  • Add candidate release notes, Proof Before Action security boundaries, compatibility/rollback guidance, and a machine-readable docs/release-state.json.
  • Keep public Action/pre-commit examples on existing tag v2.4.0 while candidate state is non-publishable.
  • Keep server.json registry metadata on published 2.4.0 until the separately reviewed release-state transition, so registry clients never receive an unavailable package version.
  • Make publishable state require finalized published/previous versions, dated changelog and comparison links, versioned release notes, and exact wheel/sdist entry points; candidate smoke installs now use the built artifacts, not unversioned PyPI.
  • Expose proof-before-action --version consistently with the sibling commands.
  • Replace automatic v* tag publication with a manual workflow that requires live non-bypassable independent environment review, an exact tag, an exact 40-character landed commit, full quality gate, clean package provenance, retained artifacts, and hash readback before the OIDC-enabled publish job.
  • Add scripts/verify_release.py and regression coverage for metadata, candidate fail-closed behavior, publication triggers, OIDC authority placement, public-version examples, and the newly used pinned artifact actions.

Checklist

  • Canonical verifier passes (uv run pytest, uv run ruff check, uv run mypy .)
  • Security implications considered — release authority, OIDC timing, dependency floor, evidence authority, rollback, and public trust claims are described below.
  • CHANGELOG.md updated under [Unreleased]
  • New or changed behavior is documented (README, docstrings, or both)

Security implications

The candidate retains mcp>=1.28.1 and immutable external Action pins. A security-focused diff review found one Medium/P2 supply-chain authorization gap in the previous workflow: pushing a v* tag automatically obtained PyPI publication authority despite the documented separate-approval boundary. This PR removes the tag trigger and makes candidate state mechanically non-publishable.

Unknown, stale, masked, unmatched, incomplete, unobservable, dirty, or authority-unverified Proof Before Action evidence remains non-authoritative. The Docker observer is not a general sandbox guarantee and intentionally accepts only bounded UTF-8 repository inputs plus explicitly named synthetic SQLite fixtures.

Residual release risks are preserved, not closed cosmetically:

  • the repository has one eligible human collaborator, so genuine independent human approval is unavailable;
  • the live pypi environment currently has no required reviewer and permits administrator bypass;
  • continuous fuzzing, OpenSSF Best Practices enrollment, and two independent redacted field reports are absent.

Those limitations do not block merging an intentionally non-publishable candidate, but public release remains NO-GO until the exact candidate is separately authorized, the environment has independent reviewer protection, and the human-review risk is satisfied or explicitly accepted.

Test plan

  • uv run pytest: 1022 passed
  • uv run pytest tests/test_proof_before_action.py -q: 125 passed
  • connector/server/release/distribution/integrity/package-policy set: 103 passed
  • Ruff, formatting, strict mypy, uv lock --check, and git diff --check: pass
  • validation corpus: 117/118, 99.2% recall; all category gates pass
  • Gitleaks whole-tree scan: no leaks
  • Clean exact commit build: wheel and sdist metadata, mcp>=1.28.1, clean producer provenance, content hygiene, embedded README, and three entry points verified
  • Isolated wheel and sdist installs: mcp-audit, mcp-audits, and proof-before-action all report 2.5.0
  • Installed Proof Before Action flow: declaration → Docker observation → comparison → trust manifest → capsule → anchored verification passed with exact producer/subject commits and independently supplied root; wrong root/image/commits, tampered HTML, malformed input, and absent authority root produced the documented fail-closed results
  • Docker cleanup: zero task-owned containers and staging images after positive and blocked paths

Exact candidate artifacts from commit 6cc0cafbc9c081bfeb876185f3d6aa7e7461db9e:

  • wheel SHA-256: 5216112e4ecd17af80e10f716cc7bad1a6ae5d317f0ee53c97303172e42c5485
  • sdist SHA-256: 22515475987c8987b61e9b6d4c8e104eaa6e49c01989341f522e8dd913518083

A fresh exact-head Codex Security scan is requested for 6cc0cafbc9c081bfeb876185f3d6aa7e7461db9e and remains required before merge; an earlier working-tree scan was correctly failed after remediation changed its snapshot.

Compatibility and rollback

2.5.0 is an additive minor release for Python 3.11–3.13 and keeps the 2.x report/SARIF compatibility policy. Existing consumers must allow mcp>=1.28.1.

Prefer reverting consuming configuration while retaining 2.5.0. If an emergency package rollback to 2.4.0 is unavoidable, also constrain mcp>=1.28.1 and read back both resolved versions; the existing v2.4.0 Action/pre-commit tag is not represented as a security-safe rollback.

Publication boundary

This PR does not authorize or perform a tag, GitHub Release, PyPI publication, deployment, or external registry/trust-source change. Finalizing release state, creating the tag, and manually dispatching publication require separate exact authorization after the landed commit and live protection state are reverified.

Related issues

N/A

@github-actions

github-actions Bot commented Jul 19, 2026

Copy link
Copy Markdown

Agent Permission Diff

  • Base: 3850b11136190a3abcee01ec923bddd7b7eb48be
  • Head: 6cc0cafbc9c081bfeb876185f3d6aa7e7461db9e
  • Findings: 1
  • Gate findings: 1
  • Acknowledged findings: 0
  • Permission changes: 18
  • Gate: observe in observe mode
  • Fail on: critical
  • Exit code: 0

Gate Decision

Max severity high is recorded but observe mode never fails.

Findings

HIGH APD002: OIDC token permission added

A workflow gained id-token: write, enabling cloud/provider identity federation.

Reviewer decision: Confirm this is paired with a protected, intended deployment path.

Evidence:

  • permissions.id-token: write

Permission Changes

  • added actions cloud_deploy curl --fail --location --silent --show-error \ --header "Accept: application/vnd.github+json" \ --header "Authorization: Bearer $GH_TOKEN... in .github/workflows/publish.yml
  • added actions cloud_deploy git fetch --force --tags origin main uv run python scripts/verify_release.py \ --tag "$RELEASE_TAG" \ --commit "$RELEASE_COMMIT" \ --envi... in .github/workflows/publish.yml
  • added actions cloud_deploy uv run python scripts/verify_release.py \ --tag "$RELEASE_TAG" \ --commit "$RELEASE_COMMIT" \ --environment-json "$RUNNER_TEMP/pypi-envir... in .github/workflows/publish.yml
  • added actions runner ubuntu-latest in .github/workflows/publish.yml
  • added actions cloud_deploy curl --fail --location --silent --show-error \ --header "Accept: application/vnd.github+json" \ --header "Authorization: Bearer $GH_TOKEN... in .github/workflows/publish.yml
  • added actions cloud_deploy pypa/gh-action-pypi-publish@ba38be9e461d3875417946c167d0b5f3d385a247 in .github/workflows/publish.yml
  • added actions deployment_environment pypi in .github/workflows/publish.yml
  • added actions runner ubuntu-latest in .github/workflows/publish.yml
  • added actions token_permission actions:read in .github/workflows/publish.yml
  • added actions token_permission contents:read in .github/workflows/publish.yml
  • added actions token_permission id-token:write in .github/workflows/publish.yml
  • added actions runner ubuntu-latest in .github/workflows/publish.yml
  • added actions token_permission actions:read in .github/workflows/publish.yml
  • removed actions cloud_deploy pypa/gh-action-pypi-publish@ba38be9e461d3875417946c167d0b5f3d385a247 in .github/workflows/publish.yml
  • removed actions deployment_environment pypi in .github/workflows/publish.yml
  • removed actions runner ubuntu-latest in .github/workflows/publish.yml
  • removed actions token_permission contents:read in .github/workflows/publish.yml
  • removed actions token_permission id-token:write in .github/workflows/publish.yml

@saagpatel

Copy link
Copy Markdown
Owner Author

@codex review

Use light reasoning. Review exact head f7d9e11c8da84a589594a34673e9fe760b8754d9, findings first and severity first. Focus on the candidate-versus-published version boundary, the removal of tag-triggered publication, exact tag/commit/main binding, candidate fail-closed behavior, OIDC authority placement, retained artifact/hash verification, immutable Action policy, rollback safety, and any misleading Proof Before Action or release claim. Do not treat unavailable review as clean.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f7d9e11c8d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread server.json
@saagpatel

Copy link
Copy Markdown
Owner Author

Agent Permission Diff disposition for exact head 37aa80f88767e16ed84dcfdb082f1aa8368710ce:

  • APD002 is an expected relocation, not a net-new repository capability: id-token: write, the pypi environment, and the pinned PyPI publisher already existed on the base publish job.
  • This candidate narrows that authority to a dedicated publish job after manual dispatch, exact tag/commit/main binding, candidate-state refusal, full quality gates, clean artifact provenance, retained-artifact transfer, and SHA-256 readback.
  • The live pypi environment still has no required reviewer and permits administrator bypass. That is not treated as protected or green: public release remains NO-GO, and the checklist requires live independent-reviewer protection before any separately authorized dispatch.
  • Candidate merge does not tag, dispatch, request OIDC, or publish; docs/release-state.json keeps the landed candidate mechanically non-publishable.

@saagpatel

Copy link
Copy Markdown
Owner Author

@codex review

Use light reasoning. Re-review exact repaired head 37aa80f88767e16ed84dcfdb082f1aa8368710ce, findings first and severity first. Confirm the prior P1 registry finding and both independent P2 package-gate findings are closed: candidate server.json must stay on public 2.4.0; publishable state must require finalized public/changelog/link/release-note metadata; the verifier must bind all three wheel/sdist entry points; and candidate install smoke must use exact built artifacts. Also recheck manual publication, tag/commit/main binding, OIDC placement, and any misleading safety or release claim. Do not treat unavailable review as clean.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 37aa80f887

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/RELEASE-CHECKLIST.md
@saagpatel

Copy link
Copy Markdown
Owner Author

@codex review

Use light reasoning. Re-review exact repaired head 65fe5519afccb3ab1e2013b1a6bb1aa20c577f84, findings first and severity first. Confirm the prior P1 registry finding, both package-gate P2s, and the checklist-order P2 are closed. In particular: candidate server.json stays on public 2.4.0; publishable state requires finalized public/changelog/link/release-note metadata; all three wheel/sdist entry points are bound; and the documented exact-candidate sequence verifies the built artifacts with the exact clean commit before installing either one. Recheck manual publication, tag/commit/main binding, OIDC placement, rollback safety, and misleading safety or release claims. Do not treat unavailable review as clean.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 65fe5519af

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/publish.yml Outdated
Comment thread scripts/verify_release.py Outdated
@saagpatel

Copy link
Copy Markdown
Owner Author

@codex review

Use light reasoning. Review exact repaired head fccd80b462aceae0d309f33676e58f3091e50b3b, findings first and severity first. Confirm all prior P1/P2 findings are closed. Focus especially on the removal of the copyable approval input, fail-closed live pypi environment readback (required reviewer, self-review prevention, admin-bypass disabled) both before build authorization and inside the OIDC publish job, finalized release-note status enforcement, exact tag/commit/main/artifact binding, and candidate non-publishability. Do not treat unavailable review as clean.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fccd80b462

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/verify_release.py Outdated
@saagpatel

Copy link
Copy Markdown
Owner Author

@codex review

Use light reasoning. Final re-review of exact head 2db2cada388e4897927e80c2d7a48081302662ee, findings first and severity first. Confirm all prior findings are closed, especially that release notes require exactly one non-conflicting status marker and exactly one matching GO/NO-GO publication decision. Recheck the live non-bypassable independent PyPI-environment gate, candidate non-publishability, tag/commit/main/artifact binding, and misleading release or safety claims. Do not treat unavailable review as clean.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2db2cada38

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/publish.yml
@saagpatel

Copy link
Copy Markdown
Owner Author

@codex review

Use light reasoning. Final exact-head review of e0cb2777ce41be416f9f664c744cf5c60ab7dc3b, findings first and severity first. Confirm all previous findings are closed, especially that manual-dispatch YAML is bound to refs/heads/main on both build and OIDC publish jobs. Recheck the live independent environment gate, exact tag/commit/main/artifact binding, singleton release-note state/decision contract, and candidate non-publishability. Do not treat unavailable review as clean.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: e0cb2777ce

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@saagpatel

Copy link
Copy Markdown
Owner Author

@codex review

Light reasoning only. Review exact head e0cb2777ce41be416f9f664c744cf5c60ab7dc3b. Findings first. This delta binds both manual publish jobs to refs/heads/main; verify that closes the prior P1 without weakening the live independent-environment, release-state, tag/commit/artifact, or candidate NO-GO gates. Do not treat unavailable review as clean.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: e0cb2777ce

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@saagpatel

Copy link
Copy Markdown
Owner Author

@codex review

Use light reasoning. Review exact head aa0144d22b86b4c58f345f2460cf330beb9af8a5, findings first. Confirm the prior non-main-dispatch P1 is closed without a green no-op: the always-running validation job must fail unless the selected workflow definition is refs/heads/main, and build must depend on it. Recheck all earlier live environment, OIDC, release-note, exact tag/commit/artifact, and candidate NO-GO gates. Do not treat unavailable review as clean.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: aa0144d22b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/publish.yml
@saagpatel

Copy link
Copy Markdown
Owner Author

@codex review

Use light reasoning. Review exact head 6cc0cafbc9c081bfeb876185f3d6aa7e7461db9e, findings first. Confirm all previous findings are closed, especially the authenticated live environment read: minimum actions: read, masked github.token passed via environment, both pre-build and OIDC-job requests authenticated, and no secret interpolation/logging. Recheck loud non-main failure and all prior publication/candidate gates. Do not treat unavailable review as clean.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: 6cc0cafbc9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@saagpatel

Copy link
Copy Markdown
Owner Author

Exact-head closeout for 6cc0cafbc9c081bfeb876185f3d6aa7e7461db9e:

  • All CI/check runs are green on this SHA.
  • All seven prior Codex review findings are repaired and their threads resolved. The final authentication delta has an independent exact-head review with no findings; the requested final Codex re-review has not returned a receipt and is not treated as clean.
  • Full local gate: 1022 passed; Ruff, formatting, mypy, lock, diff, and Gitleaks pass.
  • Exact artifacts reproduced twice: wheel 5216112e4ecd17af80e10f716cc7bad1a6ae5d317f0ee53c97303172e42c5485; sdist 22515475987c8987b61e9b6d4c8e104eaa6e49c01989341f522e8dd913518083.
  • Installed Proof Before Action exact-head lane is green, including anchored verification and fail-closed wrong-root behavior.
  • Public release remains mechanically NO-GO: live pypi environment has no reviewer rule and permits admin bypass; workflow now fails on that state.
  • Candidate merge remains blocked on the required independent GitHub approval (repository has one collaborator) and an unstarted exact-head native Codex Security workspace. No branch-protection bypass, tag, release, PyPI publication, dispatch, or registry change was performed.

@saagpatel
saagpatel merged commit 7522a91 into main Jul 19, 2026
7 checks passed
@saagpatel
saagpatel deleted the agent/release-2.5.0 branch July 19, 2026 21:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant