Refactor pytest CI architecture with parallel testing and coverage aggregation

[Copilot]

Refactor pytest CI architecture with parallel testing and coverage aggregation

Description

Refactored CI to run tests in parallel for each Python version × driver combination with isolated databases and proper coverage report aggregation. Previously, tests ran in parallel with a single shared database, risking conflicts and coverage upload race conditions.

Changes

pytest CLI arguments:

• --driver: Run tests for a single driver (psycopg2, asyncpg, pg8000, psycopg, psycopg2cffi)
• --db-name: Use a custom database name

GitHub Actions workflow:

• Matrix strategy: 4 Python versions × 5 drivers = 20 parallel test jobs
• Unique database per combination (e.g., pgmq_py312_psycopg2)
• Parallel execution for faster CI (no sequential bottleneck)

Coverage report handling:

• Each test job generates uniquely named coverage file (coverage-py{version}-{driver}.xml)
• Coverage files uploaded as GitHub artifacts with 1-day retention
• Separate upload-coverage job waits for all tests to complete
• Single batch upload to Codecov prevents race conditions

Test filtering:

• Skips non-matching drivers when --driver specified
• Uses explicit fixture name matching (not fragile string matching)

Example usage:

# Run tests for a specific driver
pytest tests --driver=psycopg2 --db-name=custom_db

# CI automatically runs all combinations in parallel

Documentation:
Updated CONTRIBUTING.md with CLI usage examples.

Status

• In progress
• Ready for review
• Done

Checklist

• Read the Contributing Guide
• Passes tests
• Linted ( we use pre-commit with ruff )
• Updated documentation

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Refactor pytest CI architecture</issue_title>
<issue_description>- Refactor the GitHub Action CI as

• For each python version
  • For each drivers
    • setup dev db
      • the db_name should be different to avoid conflict when running against same db
      • run test sequentially
• pytest script should be able to accept current driver argument, current db_name</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Refactor pytest CI architecture #17

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jason810496]

@copilot

There are race condition when uploading coverage report that cause error.
I think we need to

1. wait until all test matrix finish
2. somehow aggregate the report ( maybe for each report should be named align with the python version and driver )
3. then upload the final report / or batch upload the report

[Copilot]

@copilot

There are race condition when uploading coverage report that cause error....

Fixed in commit 2eaaf67. The workflow now:

1. Each test matrix job generates a uniquely named coverage file (coverage-py{version}-{driver}.xml)
2. Coverage files are uploaded as artifacts with 1-day retention
3. A separate upload-coverage job waits for all test jobs to complete using needs: test
4. The upload job downloads all coverage artifacts and uploads them to Codecov in a single batch

This eliminates the race condition by ensuring only one job uploads to Codecov after all tests complete.