Skip to content

Add x402 integration tests to CI#583

Open
esreekarreddy wants to merge 1 commit into
GetBindu:mainfrom
esreekarreddy:sreekar/x402-integration-ci
Open

Add x402 integration tests to CI#583
esreekarreddy wants to merge 1 commit into
GetBindu:mainfrom
esreekarreddy:sreekar/x402-integration-ci

Conversation

@esreekarreddy

@esreekarreddy esreekarreddy commented Jun 19, 2026

Copy link
Copy Markdown

Summary

Describe the problem and fix in 2-5 bullets:

  • Problem: deterministic x402 integration scenarios existed, but PR CI only ran the gRPC integration suite.
  • Why it matters: payment lifecycle regressions around settlement failure, timeout, double-spend, and replay handling could land without running the existing x402 coverage.
  • What changed: added x402/integration/e2e markers, added a dedicated deterministic x402 CI job, documented x402 integration commands, kept live-network SKALE smoke tests opt-in, and included one EOF-only pre-commit fix.
  • What did NOT change (scope boundary): no payment runtime behavior, facilitator behavior, live-network tests, storage, scheduler, or production code paths were changed.

Change Type (select all that apply)

  • Bug fix
  • Feature
  • Refactor
  • Documentation
  • Security hardening
  • Tests
  • Chore/infra

Scope (select all touched areas)

  • Server / API endpoints
  • Extensions (DID, x402, etc.)
  • Storage backends
  • Scheduler backends
  • Observability / monitoring
  • Authentication / authorization
  • CLI / utilities
  • Tests
  • Documentation
  • CI/CD / infra

Linked Issue/PR

User-Visible / Behavior Changes

None. This only changes test marker metadata, CI coverage, and test documentation.

Security Impact (required)

  • New permissions/capabilities? (No)
  • Secrets/credentials handling changed? (No)
  • New/changed network calls? (No)
  • Database schema/migration changes? (No)
  • Authentication/authorization changes? (No)
  • If any Yes, explain risk + mitigation: N/A

Verification

Environment

  • OS: macOS
  • Python version: 3.12.9
  • Storage backend: in-memory test storage
  • Scheduler backend: mocked/in-process test scheduler boundary

Steps to Test

  1. uv sync --locked --all-extras --dev
  2. uv run pytest tests/integration/x402/ -v -m "e2e and x402 and not network" --timeout=60
  3. uv run pre-commit run --files .github/workflows/ci.yml tests/integration/README.md tests/integration/x402/test_e2e_scenarios.py tests/integration/x402/test_skale_facilitator_supported.py
  4. uv run pre-commit run --all-files

Expected Behavior

  • deterministic x402 E2E tests are selected and pass
  • live-network SKALE smoke tests remain excluded from the deterministic CI selector
  • pre-commit hooks pass for the changed files and full repository hook set

Actual Behavior

  • deterministic x402 E2E tests: 4 passed, 2 deselected
  • targeted pre-commit hooks: passed
  • full pre-commit hook set: passed

Evidence (attach at least one)

  • Failing test before + passing after
  • Test output / logs
  • Screenshot / recording
  • Performance metrics (if relevant)

Human Verification (required)

What you personally verified (not just CI):

  • Verified scenarios: local x402 marker selection, deterministic payment scenario execution, live-network test exclusion, targeted pre-commit hooks, and full pre-commit hook set.
  • Edge cases checked: network-marked SKALE tests are not selected by e2e and x402 and not network; CodeRabbit checkout hardening was applied only to the new x402 job.
  • What you did NOT verify: live SKALE facilitator behavior, full repository unit test suite, and full CI matrix.

Compatibility / Migration

  • Backward compatible? (Yes)
  • Config/env changes? (No)
  • Database migration needed? (No)
  • If yes, exact upgrade steps: N/A

Failure Recovery (if this breaks)

  • How to disable/revert this change quickly: revert this PR or remove the e2e-x402-tests workflow job.
  • Files/config to restore: .github/workflows/ci.yml, tests/integration/README.md, tests/integration/x402/test_e2e_scenarios.py, tests/integration/x402/test_skale_facilitator_supported.py.
  • Known bad symptoms reviewers should watch for: x402 CI job selecting live-network tests unexpectedly, marker expression selecting zero tests, or workflow YAML validation failure.

Risks and Mitigations

  • Risk: the new CI job increases PR runtime.
    • Mitigation: it runs only four deterministic in-process scenarios and excludes live-network tests.
  • Risk: live-network SKALE smoke tests could be selected accidentally.
    • Mitigation: the CI marker expression explicitly uses not network, and the SKALE tests remain gated by X402_NETWORK_TESTS=1.

Checklist

  • Tests pass (uv run pytest)
  • Pre-commit hooks pass (uv run pre-commit run --all-files)
  • Documentation updated (if needed)
  • Security impact assessed
  • Human verification completed
  • Backward compatibility considered

Summary by CodeRabbit

  • Tests

    • Added a CI job to run deterministic x402 end-to-end integration tests using marker-based (non-network) selection with a timeout.
    • Standardized x402 E2E and SKALE facilitator test filtering with module-level pytest markers and opt-in network gating.
  • Documentation

    • Updated the integration testing guide with clearer commands for grpc-only, deterministic non-network x402, and opt-in network smoke tests, plus revised examples and troubleshooting.
  • Examples

    • Fixed the medical agent script entrypoint to bind the configured handler when executed directly.

@coderabbitai

coderabbitai Bot commented Jun 19, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 878878e7-9c5a-43e0-bc01-f7ee75eea6b1

📥 Commits

Reviewing files that changed from the base of the PR and between 119e8c4 and f70ce95.

📒 Files selected for processing (5)
  • .github/workflows/ci.yml
  • examples/medical_agent/medical_agent.py
  • tests/integration/README.md
  • tests/integration/x402/test_e2e_scenarios.py
  • tests/integration/x402/test_skale_facilitator_supported.py
✅ Files skipped from review due to trivial changes (3)
  • examples/medical_agent/medical_agent.py
  • tests/integration/x402/test_e2e_scenarios.py
  • tests/integration/x402/test_skale_facilitator_supported.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/workflows/ci.yml

📝 Walkthrough

Walkthrough

A new CI job e2e-x402-tests is added to run deterministic x402 integration tests using the marker filter "e2e and x402 and not network". Pytest markers are wired onto both x402 test modules to match that filter, and the integration test README is updated with new run commands, fixture guidance, and directory templates.

Changes

x402 E2E test marker wiring and CI job

Layer / File(s) Summary
pytest marker wiring on x402 test modules
tests/integration/x402/test_e2e_scenarios.py, tests/integration/x402/test_skale_facilitator_supported.py
test_e2e_scenarios.py gains a pytestmark list with integration, e2e, slow, and x402. test_skale_facilitator_supported.py expands its skip-only pytestmark to also include integration, x402, and network, and drops the now-redundant per-test @pytest.mark.network decorator.
New e2e-x402-tests CI job
.github/workflows/ci.yml
Adds a job depending on unit-tests that checks out the repo, installs Python 3.13 via uv, syncs dependencies, and runs pytest tests/integration/x402/ with marker filter "e2e and x402 and not network" and a 60-second timeout.
Integration test README updates
tests/integration/README.md
Adds run commands for deterministic and opt-in network x402 tests, documents the x402/ directory structure, updates gRPC fixture guidance from grpc_setup to grpc_server + mock_agent, adds a payments/ area directory template, updates the CI diagram label, and revises the troubleshooting hang command.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Poem

🐇 Hop hop, the tests now wear their marks with pride,
x402 and e2e pinned side by side,
CI spins up, the workflow grows one more job,
No network needed — just an in-process throb,
The README updated, new commands aligned,
A rabbit approves: clean wiring, well-signed! ✅

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: adding x402 integration tests to CI pipeline with deterministic test markers and a dedicated workflow job.
Description check ✅ Passed The description follows the template structure and provides comprehensive coverage including problem statement, change summary, verification steps with actual test results, security impact assessment, and risk mitigation.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@esreekarreddy esreekarreddy marked this pull request as ready for review June 19, 2026 23:41

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
.github/workflows/ci.yml (1)

89-116: ⚡ Quick win

Consider setting persist-credentials: false on the checkout action.

The checkout action at line 98 does not explicitly set persist-credentials: false. Setting this prevents the GitHub token from persisting in the workspace, reducing the risk of credential leakage if subsequent steps are compromised.

🔒 Proposed fix
     steps:
       - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683  # v4.2.2
+        with:
+          persist-credentials: false
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/ci.yml around lines 89 - 116, In the e2e-x402-tests job,
locate the actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 step and
add a with section containing persist-credentials: false to prevent the GitHub
token from persisting in the workspace after checkout, reducing the risk of
credential leakage if subsequent steps are compromised.

Source: MCP tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In @.github/workflows/ci.yml:
- Around line 89-116: In the e2e-x402-tests job, locate the
actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 step and add a with
section containing persist-credentials: false to prevent the GitHub token from
persisting in the workspace after checkout, reducing the risk of credential
leakage if subsequent steps are compromised.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 30673f1d-bd34-4b47-baa0-7f73a88ea27c

📥 Commits

Reviewing files that changed from the base of the PR and between e9e82b9 and 1a82a40.

📒 Files selected for processing (4)
  • .github/workflows/ci.yml
  • tests/integration/README.md
  • tests/integration/x402/test_e2e_scenarios.py
  • tests/integration/x402/test_skale_facilitator_supported.py

@esreekarreddy esreekarreddy force-pushed the sreekar/x402-integration-ci branch from 1a82a40 to ae5da97 Compare June 20, 2026 05:54

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (3)
tests/integration/README.md (3)

106-106: 💤 Low value

Consider rephrasing to avoid the overused word "exactly".

The phrase "This is exactly what a real SDK does" can be rephrased for stylistic variety. For example: "This mirrors what a real SDK does" or "This is how a real SDK behaves."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/integration/README.md` at line 106, The phrase "This is exactly what a
real SDK does" uses the overused word "exactly" which can be replaced with more
stylistically varied alternatives. Rephrase the sentence at line 106 in the
README.md file by substituting "exactly" with alternatives such as "mirrors"
(resulting in "This mirrors what a real SDK does") or "is how" (resulting in
"This is how a real SDK behaves") to improve the prose while maintaining the
same meaning and context about how the SDK receives messages over gRPC, runs
handlers, and returns responses.

Source: Linters/SAST tools


162-162: 💤 Low value

Specify language for fenced code block.

Line 162 should include a language identifier (e.g., ```bash instead of ```) to improve readability and syntax highlighting.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/integration/README.md` at line 162, The fenced code block starting at
line 162 is missing a language identifier for syntax highlighting. Add a
language identifier (such as bash) immediately after the opening triple
backticks in the markdown fence to enable proper syntax highlighting and improve
readability. Change the opening from ` ``` ` to ` ```bash ` or the appropriate
language for the content being displayed.

Source: Linters/SAST tools


159-159: 💤 Low value

Capitalize "GitHub" per official branding.

The software platform name should be "GitHub" (with capital "H") rather than "github".

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/integration/README.md` at line 159, In the README.md file around line
159, ensure the GitHub platform name is capitalized as "GitHub" (with a capital
"H") per official branding guidelines. Check the sentence mentioning the CI
workflow and verify that any references to the GitHub platform or GitHub Actions
use proper capitalization, while file path references like
.github/workflows/ci.yml maintain their original case as they represent actual
directory and file names in the repository.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/integration/README.md`:
- Around line 100-104: Update the MockAgentHandler code example in the README
(lines 96–104) to include the defensive checks and test verification logic
present in the actual implementation. Specifically, add a conditional check
before accessing the last message (similar to the pattern if request.messages
else None) to safely handle empty message lists, and include the
self.calls.append(request) statement that is used for test verification in the
real HandleMessages method. This will ensure the documented example matches the
complete, safe implementation and prevents IndexError issues when developers
follow the documentation.

---

Nitpick comments:
In `@tests/integration/README.md`:
- Line 106: The phrase "This is exactly what a real SDK does" uses the overused
word "exactly" which can be replaced with more stylistically varied
alternatives. Rephrase the sentence at line 106 in the README.md file by
substituting "exactly" with alternatives such as "mirrors" (resulting in "This
mirrors what a real SDK does") or "is how" (resulting in "This is how a real SDK
behaves") to improve the prose while maintaining the same meaning and context
about how the SDK receives messages over gRPC, runs handlers, and returns
responses.
- Line 162: The fenced code block starting at line 162 is missing a language
identifier for syntax highlighting. Add a language identifier (such as bash)
immediately after the opening triple backticks in the markdown fence to enable
proper syntax highlighting and improve readability. Change the opening from `
``` ` to ` ```bash ` or the appropriate language for the content being
displayed.
- Line 159: In the README.md file around line 159, ensure the GitHub platform
name is capitalized as "GitHub" (with a capital "H") per official branding
guidelines. Check the sentence mentioning the CI workflow and verify that any
references to the GitHub platform or GitHub Actions use proper capitalization,
while file path references like .github/workflows/ci.yml maintain their original
case as they represent actual directory and file names in the repository.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: bb0c9b25-a79e-4304-bb35-ab7cc1f5a01b

📥 Commits

Reviewing files that changed from the base of the PR and between 1a82a40 and ae5da97.

📒 Files selected for processing (5)
  • .github/workflows/ci.yml
  • examples/medical_agent/medical_agent.py
  • tests/integration/README.md
  • tests/integration/x402/test_e2e_scenarios.py
  • tests/integration/x402/test_skale_facilitator_supported.py
✅ Files skipped from review due to trivial changes (1)
  • examples/medical_agent/medical_agent.py
🚧 Files skipped from review as they are similar to previous changes (3)
  • .github/workflows/ci.yml
  • tests/integration/x402/test_e2e_scenarios.py
  • tests/integration/x402/test_skale_facilitator_supported.py

Comment thread tests/integration/README.md Outdated
@esreekarreddy esreekarreddy force-pushed the sreekar/x402-integration-ci branch from ae5da97 to 119e8c4 Compare June 20, 2026 05:58

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tests/integration/README.md (1)

150-157: ⚡ Quick win

Update template example filename to be generic for new payment areas.

The new payments/ directory template (lines 154–156) shows test_x402_e2e.py as the example filename, but the comment describes it as "New payment area, if separate from x402." This is contradictory: if it's a separate payment area from x402, the test file shouldn't be named test_x402_e2e.py.

Use a generic filename like test_e2e.py or test_payment_e2e.py so developers don't create misnamed files when following the template.

📝 Proposed fix for template clarity
  tests/integration/
    grpc/           # Existing
    x402/           # Existing payment flows
    payments/       # New payment area, if separate from x402
      __init__.py
-     test_x402_e2e.py
+     test_e2e.py
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/integration/README.md` around lines 150 - 157, In the directory
structure template in tests/integration/README.md, the payments/ section
describes it as a "New payment area, if separate from x402" but the example
filename is test_x402_e2e.py, which contradicts the intent of being separate
from x402. Update the example filename in the template from test_x402_e2e.py to
a generic filename like test_e2e.py or test_payment_e2e.py so developers
following this template won't create misnamed test files when creating new
payment areas.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/integration/README.md`:
- Around line 150-157: In the directory structure template in
tests/integration/README.md, the payments/ section describes it as a "New
payment area, if separate from x402" but the example filename is
test_x402_e2e.py, which contradicts the intent of being separate from x402.
Update the example filename in the template from test_x402_e2e.py to a generic
filename like test_e2e.py or test_payment_e2e.py so developers following this
template won't create misnamed test files when creating new payment areas.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: dc86e067-a8e8-4c7b-9e80-736752729a40

📥 Commits

Reviewing files that changed from the base of the PR and between ae5da97 and 119e8c4.

📒 Files selected for processing (5)
  • .github/workflows/ci.yml
  • examples/medical_agent/medical_agent.py
  • tests/integration/README.md
  • tests/integration/x402/test_e2e_scenarios.py
  • tests/integration/x402/test_skale_facilitator_supported.py
✅ Files skipped from review due to trivial changes (3)
  • tests/integration/x402/test_e2e_scenarios.py
  • examples/medical_agent/medical_agent.py
  • tests/integration/x402/test_skale_facilitator_supported.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/workflows/ci.yml

@esreekarreddy esreekarreddy force-pushed the sreekar/x402-integration-ci branch from 119e8c4 to f70ce95 Compare June 20, 2026 06:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant