Skip to content

CI: split ci-cd.yml into tiers so docs/AI-only PRs skip the full matrix #1557

@thehesiod

Description

@thehesiod

Problem

ci-cd.yml currently runs the full matrix (6 Python versions × 2
OS × several variants = ~13+ test jobs, plus build, zizmor, coverage,
check) on every PR, regardless of what changed. That includes:

  • docs-only changes (docs/**, *.md, *.rst)
  • .github/workflows/claude.yml + prompt edits (.github/claude-*-prompt.md,
    plugins/aiobotocore-bot/**) — the AI-automation code path
  • .github/workflows/*.yml edits that don't affect the library
  • CHANGES.rst, CONTRIBUTING.rst, .pre-commit-config.yaml,
    pyproject.toml [tool.rumdl], .serena/, etc.

Running the full matrix for "fix a typo in docs/override-patterns.md"
burns ~15 CI minutes and blocks the merge queue for the sake of a
bytecode path the change didn't touch.

Proposal

Split CI into tiers driven by on.pull_request.paths / paths-ignore,
with a single umbrella check (check job / alls-green) that
aggregates per-tier results so the merge-queue status check name stays
stable.

Tier A — docs-only changes

Triggered when only paths in this set change:

  • docs/**
  • **.md, **.rst (except CHANGES.rst when aiobotocore/__init__.py
    also changed — treat as code then)
  • CONTRIBUTING.rst, README.rst

Runs:

  • pre-commit run --all (lint/format via ruff, yamllint, rumdl)
  • make docs (Sphinx build)

Skips the Python test matrix entirely.

Tier B — AI automation / workflow / tooling changes

Triggered when only these change:

  • .github/workflows/claude.yml
  • .github/workflows/botocore-sync.yml
  • .github/claude-*-prompt.md
  • plugins/aiobotocore-bot/**
  • .claude-plugin/**
  • .github/usage-summary.py
  • .github/ISSUE_TEMPLATE/**, .github/pull_request_template.md
  • .pre-commit-config.yaml, pyproject.toml (only [tool.rumdl],
    [tool.ruff], [tool.uv], [tool.pytest.ini_options] sections —
    may be hard to detect precisely; fall back to Tier C if any non-tool
    section changed)
  • .github/dependabot.yml, .codecov.yml, .yamllint, .readthedocs.yml

Runs:

  • pre-commit run --all
  • zizmor
  • A single Python job (one version, one OS, core path only) to
    confirm the workflow/tooling changes didn't break the basic test
    harness. Not the full matrix.

Tier C — library code / dependency changes

Triggered when aiobotocore/**, tests/**, pyproject.toml
(non-[tool.*]), uv.lock, or anything else changes — this is the
default path and runs today's full matrix.

Implementation sketch

  • Each tier becomes its own workflow (or one workflow with multiple
    top-level jobs + paths: filters).
  • alls-green / check job requires whichever tier actually fired.
    Use allowed-skips in re-actors/alls-green@… to accept skipped
    Tier-C matrix when Tier A fired, and so on.
  • The merge-queue required-status-check remains a single check name
    so branch protection rules don't need to change.

Alternative (simpler, less pure): use paths-ignore: on the existing
workflow to skip Tier-C on docs/AI paths, and add a lightweight
dedicated docs.yml / ai.yml for the other tiers. Same effect, less
refactoring.

Motivation

PR #1556 ran the full CI matrix ~7 times during its back-and-forth —
each run ~15 minutes of CI time — for a PR that is 99% documentation
and automation config with no library code changes. Across all the AI
workflow changes landing lately, this has amounted to hours of pointless
matrix runs.

The zizmor rate-limit failure we just debugged was partly caused by
setup-uv fetching releases on every one of those full-matrix runs;
reducing matrix runs for docs/AI PRs also reduces API-budget pressure.

Non-goals

  • Don't reduce coverage for actual library changes. Tier C stays the
    current full matrix.
  • Don't change test semantics. Just change when each tier fires.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions