Skip to content

ci: add minimal-version resolution matrix to cpu and gpu jobs#214

Open
janickm wants to merge 1 commit into
NVIDIA:mainfrom
janickm:dev/janickm/ci-minimal-version-matrix
Open

ci: add minimal-version resolution matrix to cpu and gpu jobs#214
janickm wants to merge 1 commit into
NVIDIA:mainfrom
janickm:dev/janickm/ci-minimal-version-matrix

Conversation

@janickm

@janickm janickm commented May 29, 2026

Copy link
Copy Markdown
Collaborator

Summary

Add a resolution: [pinned, minimal] matrix dimension to both CI jobs, validating that declared minimum dependency bounds actually work.

How it works

Matrix cell Resolution strategy What it validates
pinned uv sync (lock file) Current behavior -- tests against exact locked versions
minimal uv sync --resolution lowest-direct Declared minimum bounds are installable and pass tests

The minimal cell re-resolves the workspace from scratch using the lowest allowed version for each direct dependency (e.g. torch>=2.9 resolves to torch==2.9.0 instead of 2.12.0). This catches cases where code uses APIs not available at the declared minimum.

Details

  • Both cells run linting (pre-commit) and unit tests
  • Wheel build only runs on pinned (we publish against locked versions)
  • publish-testpypi requires all matrix cells (needs: [cpu, gpu]) -- both pinned and minimal must pass before publishing
  • fail-fast: false -- both cells run to completion even if one fails
  • Cache keys are separated: cpu-pinned / cpu-minimal, gpu-sm{arch}-pinned / gpu-sm{arch}-minimal

Prerequisites

Depends on #211 which pins lower bounds on all workspace dependencies (enabling --resolution lowest-direct to resolve without failures).

Local validation

Tested locally with torch 2.9.0 (lowest-direct resolution):

  • CPU tests: 50 passed
  • GPU tests: 21 passed (RTX A6000)

@janickm janickm marked this pull request as draft May 29, 2026 13:39
@copy-pr-bot

copy-pr-bot Bot commented May 29, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@janickm janickm self-assigned this May 29, 2026
@janickm janickm changed the base branch from main to dev/janickm/lower-torch-req May 29, 2026 13:42
@janickm janickm force-pushed the dev/janickm/ci-minimal-version-matrix branch from 5c67b3b to 26ae181 Compare May 29, 2026 13:48
@janickm janickm marked this pull request as ready for review May 29, 2026 13:50
@janickm janickm changed the base branch from dev/janickm/lower-torch-req to main May 29, 2026 13:52
@janickm janickm force-pushed the dev/janickm/ci-minimal-version-matrix branch 11 times, most recently from 2bcac5c to aadb9ad Compare May 29, 2026 16:34
Add a resolution: [pinned, minimal] matrix dimension to both CI jobs.

- pinned: uses uv sync from the lock file (existing behavior), runs
  lint (pre-commit/ty) and wheel build in addition to tests.
- minimal: uses uv sync --resolution lowest-direct, which re-resolves
  to the lowest versions allowed by pyproject.toml bounds (e.g. torch
  2.9 instead of 2.12). Only runs tests -- linting is skipped because
  older type stubs produce false positives against current code.

fail-fast: false so both cells run to completion even if one fails.
publish-testpypi requires all matrix cells to pass.

Also:
- Removes the Windows-only cu130 PyTorch index and triton-windows dep
  from flashdreams/pyproject.toml. Windows CUDA users can configure
  their own index. Removing the cu130 index eliminates cross-index
  CUDA version mismatches during lowest-direct resolution.
- Lowers omnidreams and ludus-renderer torch floor from >=2.11 to >=2.9.
- Lowers omnidreams torchvision from >=0.26 to >=0.24.
- Raises opencv-python-headless from >=4.5 to >=4.6.
- Raises pytest-asyncio from >=0.23 to >=0.24.
@janickm janickm force-pushed the dev/janickm/ci-minimal-version-matrix branch from aadb9ad to 94ab942 Compare May 29, 2026 16:43
@janickm janickm added enhancement New feature or request ci Build / CI-related labels May 31, 2026
@greptile-apps

greptile-apps Bot commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds a resolution: [pinned, minimal] strategy matrix to both the cpu and gpu CI jobs, with the minimal cell re-resolving dependencies via uv sync --resolution lowest-direct to validate declared minimum bounds. Lint and wheel build steps are correctly gated to the pinned cell; cache keys are cleanly separated; and publish-testpypi gates on all matrix cells passing.

  • The lock file itself was regenerated with --resolution lowest-direct, writing resolution-mode = \"lowest-direct\" into uv.lock [options]. This makes the mode sticky for all future uv lock invocations, so both the pinned and minimal cells now install the same version set — the CONTRIBUTING.md promise of "pinned versions from uv.lock" no longer refers to latest-compatible locked versions.
  • flashdreams/pyproject.toml also drops Windows-specific overrides (triton-windows, the pytorch-cu130 index, and win32 PyTorch source routing), and the lock file drops all Windows platform markers.
  • Several package lower bounds are bumped workspace-wide (opencv-python-headless 4.5→4.6, pytest-asyncio 0.23→0.24, torchvision 0.26→0.24, torch 2.11→2.9 in two integrations).

Confidence Score: 4/5

Safe to merge after confirming whether the lock file being permanently lowest-direct is intentional.

The CI plumbing is sound — matrix expansion, cache key separation, conditional lint/wheel steps, and publish gating all work as described. The uv.lock gaining resolution-mode = lowest-direct in its options block makes every future uv lock call continue resolving to lowest-direct, meaning both CI cells install identical package versions with no coverage of latest compatible versions. The ambiguity around intent is what keeps this from a clean score.

uv.lock — specifically the new [options] resolution-mode entry and the loss of all Windows platform markers.

Important Files Changed

Filename Overview
.github/workflows/ci.yml Adds resolution matrix to both cpu and gpu jobs; skips lint and wheel build for minimal; cache keys correctly separated; one minor inefficiency in minimal CPU install
uv.lock Regenerated with --resolution lowest-direct, adding [options] resolution-mode = "lowest-direct" which makes the lock file sticky to lowest-direct; also drops all Windows platform markers; "pinned" and "minimal" CI cells now resolve the same dependency versions
flashdreams/pyproject.toml Removes Windows-specific triton-windows dep and pytorch-cu130 index override; bumps opencv lower bound from 4.5 to 4.6
pyproject.toml Bumps pytest-asyncio minimum from 0.23 to 0.24 in test and lint dependency groups
CONTRIBUTING.md Updates CI documentation to reflect both pinned and minimal resolution strategies; accurate if lock file keeps non-lowest-direct semantics, slightly misleading if lock file is sticky to lowest-direct
integrations/omnidreams/pyproject.toml Lowers torch minimum from 2.11 to 2.9 and torchvision from 0.26 to 0.24, consistent with workspace-wide torch floor; bumps opencv lower bound to 4.6

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    push[Push / merge_group] --> cpu & gpu

    subgraph cpu["cpu job (matrix: pinned | minimal)"]
        C1[Install system deps] --> C2[Checkout + setup uv]
        C2 --> C3{matrix.resolution}
        C3 -- pinned --> C4["uv sync --extra dev --group lint"]
        C3 -- minimal --> C5["uv sync --resolution lowest-direct\n--extra dev --group lint"]
        C4 --> C6["Run linter checks (pinned only)"]
        C4 --> C7["Build wheel (pinned only)"]
        C5 -. skipped .-> C6
        C5 -. skipped .-> C7
        C6 --> C8[pytest -m ci_cpu]
        C7 --> C8
        C5 --> C8
    end

    subgraph gpu["gpu job (matrix: pinned | minimal)"]
        G1[Detect GPU arch] --> G2[Checkout + setup uv]
        G2 --> G3{matrix.resolution}
        G3 -- pinned --> G4["uv sync --extra dev"]
        G3 -- minimal --> G5["uv sync --resolution lowest-direct --extra dev"]
        G4 --> G6[pytest -m ci_gpu]
        G5 --> G6
        G6 --> G7[Trim uv cache]
    end

    cpu & gpu --> pub{all 4 cells pass?}
    pub -- yes, main branch --> PYPI[publish-testpypi]
    pub -- no --> FAIL[Block publish]
Loading

Comments Outside Diff (1)

  1. uv.lock, line 373-374 (link)

    P2 Lock file pinned to lowest-direct — "pinned" and "minimal" cells test identical versions

    The resolution-mode = "lowest-direct" entry in the lock file's [options] block is sticky: every future uv lock invocation (dependency updates, bots) will continue resolving to lowest-direct. This means the "pinned" cell (uv sync, which installs what's in the lock file) and the "minimal" cell (uv sync --resolution lowest-direct, which re-resolves) resolve to the same package versions. Concretely, the lock file now pins aiohttp to 3.9.0 instead of the previously locked 3.13.5. There is no longer any CI coverage against the latest compatible versions.

    If the intent is for the lock file to represent the minimum viable baseline, the CONTRIBUTING.md wording ("pinned versions from uv.lock") could be clarified to "pinned minimum versions". If the intent is for pinned to continue tracking latest compatible versions, the lock file should be regenerated without --resolution lowest-direct (removing this [options] block).

Reviews (1): Last reviewed commit: "ci: add minimal-version resolution matri..." | Re-trigger Greptile

Comment thread .github/workflows/ci.yml
Comment on lines +70 to +73
if [ "${{ matrix.resolution }}" = "minimal" ]; then
uv sync --resolution lowest-direct --extra dev --group lint \
--no-install-package transformer-engine-torch
else

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The minimal CPU cell installs --group lint (which pulls in pre-commit and ty) even though the "Run linter checks" step is skipped for that cell. Only --group test is needed to satisfy pytest-asyncio and imageio-ffmpeg for the unit-test step, saving two unnecessary package installs per minimal run.

Suggested change
if [ "${{ matrix.resolution }}" = "minimal" ]; then
uv sync --resolution lowest-direct --extra dev --group lint \
--no-install-package transformer-engine-torch
else
if [ "${{ matrix.resolution }}" = "minimal" ]; then
uv sync --resolution lowest-direct --extra dev --group test \
--no-install-package transformer-engine-torch
else

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Build / CI-related enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant