ci(tests): shard the server suite across a 4-runner matrix

.github/workflows/ci.yml

@@ -15,6 +15,17 @@
 #     its budget, so new heavies must either get the Slow trait or a real fix.
 # See the Test step below for details.
 #
+# SHARDED TEST MATRIX (#716): the server suite is CPU-bound and already scales ~perfectly with cores
+# (summed test time ≈ 2550 s fast-tier; at maxParallelThreads 4 on the 4-vCPU runner that IS the
+# ~11 min wall the PR gate used to spend). More in-process threads would gain nothing (no more cores;
+# xunit.runner.json pins 4 because of the WorldGenerator lock convoy, #536) — so the suite fans out
+# over 4 runners instead, cutting the test step to ~3 min. scripts/partition-tests.py deterministically
+# packs test CLASSES onto shards (weights: scripts/test-shard-weights.json, from real trx data) and
+# emits each shard's --filter. Shard 1 additionally runs the small Client.Tests suite (~11 s) and a
+# partition-completeness check (--list-tests cross-check) so a class no shard filter matches can never
+# be SILENTLY untested. Branch protection requires the `tests-passed` fan-in job, NOT the matrix jobs
+# (matrix job names carry the shard index and a skipped required matrix would not report).
+#
 # We target the test projects directly (not the whole .sln) on purpose: the .sln also contains the
 # WinForms launcher (net10.0-windows), which cannot build on the Linux runner. Building the test
 # projects pulls in exactly their dependencies (server/shared/client.core) and nothing Windows-only.
@@ -26,10 +37,12 @@
 # REQUIRED-CHECK SAFE doc/screenshot-only skip: rather than skipping the whole workflow with paths-ignore
 # (which would leave a *required* check stuck on "Expected — Waiting for status" for a docs-only PR), the
 # workflow always runs and a `changes` job decides whether the heavy `build-test`/`format` jobs run. On a
-# doc- or image-only PR they are skipped via their `if:`, and GitHub counts a skipped required job as a
-# pass — so the `Build + test (.NET, headless)` check still reports green and never blocks the PR. Any
-# code/content file in the diff makes them run for real. The doc/image skip list lives in the shared
-# reusable workflow .github/workflows/detect-doc-only.yml (also used by lint.yml and codeql.yml).
+# doc- or image-only PR the build-test shards are skipped via their `if:`, and the `tests-passed` fan-in
+# explicitly treats "changes succeeded + build-test skipped" as a pass — so the required check reports
+# green and never blocks a docs PR. Any code/content file in the diff makes them run for real. The
+# doc/image skip list lives in the shared reusable workflow .github/workflows/detect-doc-only.yml (also
+# used by lint.yml and codeql.yml). The `format` job stays a directly-required check (its name is stable
+# and GitHub counts its skip as a pass, as before).
 
 name: CI
 
@@ -54,12 +67,16 @@ jobs:
     uses: ./.github/workflows/detect-doc-only.yml
 
   build-test:
-    name: Build + test (.NET, headless)
+    name: Build + test (shard ${{ matrix.shard }}/4)
     needs: changes
     # Run for any push to main, or for a PR that touched at least one non-doc file. A docs-only PR
-    # skips this job; a skipped required check counts as passing, so docs PRs are never blocked.
+    # skips this job; the `tests-passed` fan-in treats that as passing, so docs PRs are never blocked.
     if: github.event_name != 'pull_request' || needs.changes.outputs.nondocs == 'true'
     runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false # one failing shard must not cancel the others' results
+      matrix:
+        shard: [1, 2, 3, 4]
     steps:
       - uses: actions/checkout@v5
 
@@ -77,30 +94,52 @@ jobs:
             **/*.csproj
             Directory.Build.props
 
+      # Shard 1 also owns the small Client.Tests suite (~11 s), so shards 2-4 skip restoring and
+      # building it entirely.
       - name: Restore
         run: |
           dotnet restore tests/BlocksBeyondTheStars.Tests/BlocksBeyondTheStars.Tests.csproj
-          dotnet restore tests/BlocksBeyondTheStars.Client.Tests/BlocksBeyondTheStars.Client.Tests.csproj
+          if [ "${{ matrix.shard }}" = "1" ]; then
+            dotnet restore tests/BlocksBeyondTheStars.Client.Tests/BlocksBeyondTheStars.Client.Tests.csproj
+          fi
 
       # -warnaserror promotes every compiler/analyzer warning to a build error for the PR gate.
       # Release configuration: the suite is dominated by CPU-bound worldgen/sim tests, which run far
       # faster optimized — the correctness gate is the same tests, just not on a Debug-slowed engine.
       - name: Build (treat warnings as errors)
         run: |
           dotnet build tests/BlocksBeyondTheStars.Tests/BlocksBeyondTheStars.Tests.csproj --no-restore -c Release -warnaserror
-          dotnet build tests/BlocksBeyondTheStars.Client.Tests/BlocksBeyondTheStars.Client.Tests.csproj --no-restore -c Release -warnaserror
+          if [ "${{ matrix.shard }}" = "1" ]; then
+            dotnet build tests/BlocksBeyondTheStars.Client.Tests/BlocksBeyondTheStars.Client.Tests.csproj --no-restore -c Release -warnaserror
+          fi
+
+      # Partition-completeness guard (shard 1 only — needs a built assembly, any shard's would do):
+      # every test that --list-tests discovers must be matched by EXACTLY one shard's filter. A test
+      # class the partition misses (nested class, name colliding with a namespace segment, …) fails
+      # the build here instead of silently never running on any shard.
+      - name: Verify shard partition covers every test
+        if: matrix.shard == 1
+        run: |
+          dotnet test tests/BlocksBeyondTheStars.Tests/BlocksBeyondTheStars.Tests.csproj --no-build -c Release --list-tests > listed-tests.txt
+          python3 scripts/partition-tests.py verify --shards 4 --list-file listed-tests.txt
 
       # Two-tier test gate: PRs skip the tests marked [Trait("Category", "Slow")] — the statistical
       # worldgen/simulation heavies that dominate the suite's CPU time. Full coverage is still
-      # guaranteed twice downstream: every push to main runs the COMPLETE suite here (empty filter),
-      # and release.yml runs it again on the tagged commit before anything is published.
+      # guaranteed twice downstream: every push to main runs the COMPLETE suite here (shard filter
+      # only, no tier filter), and release.yml runs it again on the tagged commit before anything is
+      # published. The shard filter (which test classes THIS runner executes) is combined with the
+      # tier filter via `&`.
       - name: Test
         env:
           # xUnit maps [Trait("Category", ...)] to the vstest Category property.
-          FILTER: ${{ github.event_name == 'pull_request' && 'Category!=Slow' || '' }}
+          TIER: ${{ github.event_name == 'pull_request' && 'Category!=Slow' || '' }}
         run: |
-          dotnet test tests/BlocksBeyondTheStars.Tests/BlocksBeyondTheStars.Tests.csproj --no-build -c Release ${FILTER:+--filter "$FILTER"} --logger "trx;LogFileName=server.trx" --results-directory TestResults
-          dotnet test tests/BlocksBeyondTheStars.Client.Tests/BlocksBeyondTheStars.Client.Tests.csproj --no-build -c Release ${FILTER:+--filter "$FILTER"} --logger "trx;LogFileName=clientcore.trx" --results-directory TestResults
+          SHARD_FILTER="$(python3 scripts/partition-tests.py filter --shard ${{ matrix.shard }} --shards 4)"
+          FILTER="${TIER:+$TIER&}$SHARD_FILTER"
+          dotnet test tests/BlocksBeyondTheStars.Tests/BlocksBeyondTheStars.Tests.csproj --no-build -c Release --filter "$FILTER" --logger "trx;LogFileName=server.trx" --results-directory TestResults
+          if [ "${{ matrix.shard }}" = "1" ]; then
+            dotnet test tests/BlocksBeyondTheStars.Client.Tests/BlocksBeyondTheStars.Client.Tests.csproj --no-build -c Release ${TIER:+--filter "$TIER"} --logger "trx;LogFileName=clientcore.trx" --results-directory TestResults
+          fi
 
       # Fast-tier duration guardrail (PRs only — the full run on main legitimately contains Slow tests):
       # fail when any non-Slow test exceeds the per-test budget, so the fast tier cannot silently decay
@@ -117,10 +156,31 @@ jobs:
         if: always()
         uses: actions/upload-artifact@v6
         with:
-          name: test-results
+          name: test-results-shard-${{ matrix.shard }}
           path: TestResults/*.trx
           if-no-files-found: ignore
 
+  # Fan-in for branch protection: the ONE required check standing in for the whole test matrix.
+  # Required checks must not point at matrix jobs directly (names carry the shard index, and a
+  # docs-only PR would leave them "Expected — Waiting for status" forever). Passing states:
+  #   * build-test == success  → all four shards green
+  #   * build-test == skipped  → docs-only PR (the `changes` job said nondocs=false) — counts as pass,
+  #     but ONLY when `changes` itself succeeded, so a cancelled/failed upstream can't sneak through.
+  tests-passed:
+    name: Tests passed
+    needs: [changes, build-test]
+    if: always()
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check matrix result
+        run: |
+          echo "changes: ${{ needs.changes.result }} / build-test: ${{ needs.build-test.result }}"
+          if [ "${{ needs.changes.result }}" != "success" ]; then exit 1; fi
+          case "${{ needs.build-test.result }}" in
+            success|skipped) exit 0 ;;
+            *) exit 1 ;;
+          esac
+
   # Verifies C# code style/formatting (whitespace, using-order, object-initializer line breaks, …) per
   # .editorconfig. Runs against BlocksBeyondTheStars.CI.slnf, a solution filter listing every project EXCEPT
   # the Windows-only WinForms launcher, so the whole set loads on the Linux runner. Line endings are LF here

TODO.md

@@ -10,6 +10,7 @@ keep it current when controls/features change. Last consolidated 2026-06-04.
 **Test:** `./scripts/run-tests.sh` — currently **1421 server + 154 client passing** (2026-08-03). Locale parity (en/de) is enforced by a test.
 CI runs two tiers: PRs skip the tests marked `[Trait("Category", "Slow")]`; pushes to `main` and the release workflow run the full suite. CI builds/runs
 tests in Release, and a per-test duration guardrail (`scripts/check-test-durations.py`, PRs only) fails the gate when a non-Slow test exceeds 120 s.
+The server suite is sharded across a 4-runner matrix (`scripts/partition-tests.py` + checked-in weights; `Tests passed` is the required fan-in check) — PR gate ~4½ min.
 **Conventions:** English docs/comments; in-game text bilingual DE+EN; commit to `main` with the
 Claude `Co-Authored-By` trailer; OpenAI texture + ElevenLabs sound generation is blanket-approved
 (no per-batch gate).
@@ -102,6 +103,19 @@ Per-item detail lives in the dated work log below. **Since 2026-07 versions are
 
 ---
 
+### ★ CI PR gate 12:40 → ~4:30: server suite sharded over a 4-runner matrix (#716, 2026-08-04, branch ci/shard-tests)
+The Test step spent 11:25 of the 12:40 PR gate running the server suite on ONE 4-vCPU runner — and it
+already scaled ~perfectly in-process (summed test time ≈ 2550 s ÷ 4 threads ≈ the observed wall;
+`maxParallelThreads` is pinned to 4 on purpose, see #536), so the only parallelization left was more
+runners. `scripts/partition-tests.py` now packs test CLASSES onto 4 shards (greedy by weight from
+`scripts/test-shard-weights.json`, real trx seconds; new classes get a default) and emits per-shard
+dot-anchored `FullyQualifiedName~.<Class>.` filters. Classes are parsed from source declarations, not
+file names, and shard 1 cross-checks `dotnet test --list-tests` so every test provably maps to exactly
+one shard — a partition miss fails the build instead of silently skipping tests. Shard 1 also carries
+the 11 s Client.Tests suite; the duration guardrail runs per shard. New `Tests passed` fan-in job is
+the single required branch-protection check (matrix names can't be; docs-only skips still pass).
+Verified locally: all 4 shard filters together run 461+376+237+324 = exactly the 1398 fast-tier tests.
+
 ### ★ Multi-biome worlds shuffle WHICH biomes they get, not just how many (#696, 2026-08-03, branch fix/biome-subset-shuffle)
 `ResolveBiomes` randomised only the biome COUNT (2..pool) and then always took the first N entries of
 the type's pool in `data/planets.json` order — so e.g. a 2-biome `varied` world was always sand+grass,

docs/developer/DEVELOPER.md

@@ -41,10 +41,15 @@ dotnet test                                # all .NET xUnit suites (server/share
 ./scripts/run-tests.sh --suites ClientCore  # just the headless client<->server integration tests
 ```
 
-CI runs these suites **two-tiered** (`.github/workflows/ci.yml`): pull requests skip the ~31 statistical /
-soak tests marked `[Trait("Category", "Slow")]` for a fast gate (~3 min instead of ~13), while every push to
-`main` — and `release.yml` before publishing — runs the complete suite. Reproduce the fast PR tier locally
-with `dotnet test --filter "Category!=Slow"`.
+CI runs these suites **two-tiered** (`.github/workflows/ci.yml`): pull requests skip the statistical /
+soak tests marked `[Trait("Category", "Slow")]` for a fast gate, while every push to `main` — and
+`release.yml` before publishing — runs the complete suite. Reproduce the fast PR tier locally with
+`dotnet test --filter "Category!=Slow"`. On top of the tiering, CI **shards the server suite across a
+4-runner matrix** (#716): the suite is CPU-bound and already scales ~perfectly with cores in-process
+(`xunit.runner.json` pins `maxParallelThreads: 4`, see #536), so the only way to go faster is more
+runners. `scripts/partition-tests.py` deterministically packs test classes onto shards (weights from
+`scripts/test-shard-weights.json` — refresh it occasionally from a CI `.trx` artifact when balance
+drifts) and shard 1 cross-checks `--list-tests` output so every test provably runs on exactly one shard.
 
 `run-tests.ps1` selects suites via `-Suites` (`Dotnet`, `ClientCore`, `UnityEdit`, `UnityPlay`, `All`); the
 Unity suites are opt-in so they don't slow the common loop, and need `Unity.exe` (pass `-UnityPath` if not at
@@ -83,20 +88,24 @@ These are the same two suites `run-tests.ps1` / `run-tests.sh` run by default. T
   (The detection is an explicit `case` match, not `dorny/paths-filter` — that action's `some`-glob semantics
   made a `**/*` + `!doc` list evaluate true for every diff, so docs PRs never actually skipped.)
 
-It is **safe as a required status check** — and is one: because the workflow always runs and the build-test
-check always reports (green/red/skipped-as-pass), a docs-only PR is never left waiting on a missing status, the
-usual failure mode of a plain `paths-ignore` skip. (Require the `Build + test (.NET, headless)` job, **not** the
+It is **safe as a required status check** via the `Tests passed` fan-in job: the build-test shards are a
+matrix (names carry the shard index, so they can't serve as required checks themselves), and `Tests passed`
+runs `if: always()`, passing exactly when all shards succeeded or were skipped as docs-only (with the
+`changes` helper itself green) — so a docs-only PR is never left waiting on a missing status, the usual
+failure mode of a plain `paths-ignore` skip. (Require `Tests passed`, **not** the shard jobs or the
 `Detect code changes` helper.)
 
 The **Unity** suites (`UnityEdit` / `UnityPlay`) are **not** in CI — they need the Editor and stay local/opt-in
 (run them with `./scripts/run-tests.ps1 -Suites All` before a client-affecting change). The release build
 ([`release.yml`](../../.github/workflows/release.yml)) is a **separate** workflow that triggers only on tags;
 it does not run tests, so the PR gate is where correctness is checked before merge.
 
-> **Required status checks on `main`** (branch protection): `Build + test (.NET, headless)`,
-> `Format (dotnet format)`, `ruff (Python ai-backend)`, `actionlint (workflows)`
-> and `CodeQL` — all five must pass before merge. The `Detect code changes` helper is intentionally **not**
-> required (it's an always-skippable gate; only require jobs that always report). `strict` is off (no forced
+> **Required status checks on `main`** (branch protection): `Tests passed`,
+> `Format (dotnet format)`, `ruff (Python ai-backend)` and `actionlint (workflows)` — all four must pass
+> before merge (CodeQL runs on PRs but is not a required check, contrary to what this doc used to
+> claim — verified against the live branch protection 2026-08-04). The `Detect code changes` helper and the individual
+> `Build + test (shard N/4)` matrix jobs are intentionally **not** required (only require jobs that always
+> report under a stable name; `Tests passed` fans the matrix in). `strict` is off (no forced
 > rebase) and `enforce_admins` is off (an owner can still `gh pr merge --admin` in an emergency).
 
 ### Other PR checks: format, lint, CodeQL