[code-infra] Run the browser tests with one page per project - #23461
[code-infra] Run the browser tests with one page per project#23461LukasTy wants to merge 2 commits into
Conversation
Vitest opens `maxWorkers` pages per project and keeps every one of them until the whole run ends (vitest-dev/vitest#10990), so the peak renderer count is `projects * maxWorkers` rather than anything bounded by the work left to do. With 21 browser projects and `maxWorkers: 2` that is 42 Chromium renderers alive at once, which is what puts the job near the memory ceiling of its container. Setting `VITEST_MAX_WORKERS=1` halves it. Measured locally on the full suite with a cold dependency cache: peak renderers 43 -> 21 total renderer footprint 11535MB -> 6702MB (-42%) wall time 175s -> ~190s (+10%) The env var is required: a `--maxWorkers` CLI flag does not reach the projects, because the per-project value from `vitest.shared.mts` wins. Vitest applies the env var last, inside each project's own config resolution. Co-Authored-By: Claude Opus 5 <[email protected]>
Deploy previewBundle size
PerformanceTotal duration: 1,612.87 ms +73.47 ms(+4.8%) | Renders: 63 (+0) | Paint: 2,210.70 ms +120.69 ms(+5.8%)
…and 10 more (+11 within noise) — details Metric alarms
…and 2 more metric alarms — details Check out the code infra dashboard for more information about this PR. |
LukasTy
added
test
type: enhancement
|
On CI the wall time cost is bigger than local. 155.9s here (186s job) vs 112-134s (141-167s job) on five master runs from today. About 25-35% change Retrying is slower though 🤷 |
|
Yeah, this is slightly more expensive per run, but if it buys us stable CI until Vitest resolves their internals, I think it would end up being net positive. |
Janpot
left a comment
Janpot
left a comment
There was a problem hiding this comment.
Have nothing against it. So far it seems stable at the moment. We could also keep this in the back pocket for when it starts flaking again. Whichever goes first, @JCQuintas PR at vitest, or MUI X adds another big feature.
|
Fair enough. We pulled a few levers now. We can keep the current approach until we see OOM issues again. 👌 |
3 participants
Vitest opens
maxWorkerspages per project and keeps every one of them alive until the whole run ends (vitest#10990). The peak renderer count is thereforeprojects * maxWorkers, not anything bounded by the work left to do. With 21 browser projects andmaxWorkers: 2that is 42 Chromium renderers alive at once, which is what putstest_browsernear the memory ceiling of its container and produces the intermittentBrowser page crashed/Browser connection was closedfailures.Setting
VITEST_MAX_WORKERS=1halves that.Measured
Full suite, cold dependency cache, same machine. Memory is
phys_footprintsummed across the renderer processes - RSS is not trustworthy here because it counts shared pages once per process, and it reported only a 10% drop for the same change.Footprint per renderer goes up, because each remaining page now runs about twice as many files and
isolateis off, so it accumulates more. Halving the process count more than compensates.On a 16 GB container this is roughly 11.5 GB -> 6.7 GB of renderers, which turns "running at the ceiling" into about 5 GB of headroom.
CircieCI results
https://app.circleci.com/pipelines/github/mui/mui-x/139278/workflows/a5ed8196-8eb3-431f-9b0d-13b66be8fece/jobs/903571/resources CI is sitting happily at ~75% RAM usage.
Looks like a decent compromise for the time being.
Why the env var and not a flag
--maxWorkers=1on the command line does not reach the projects: the per-project value fromvitest.shared.mtswins, so the flag looks like a no-op. Vitest appliesVITEST_MAX_WORKERSlast, inside each project's own config resolution, so it is the one that propagates.Same four projects (
x-charts*), same run, for illustration:VITEST_MAX_WORKERS=1maxWorkers: 2)--maxWorkers=1VITEST_MAX_WORKERS=7Worth noting the last row: higher values are much worse, roughly 6 renderers per project.
Scope
One env var on the
test_browserjob, so it coverstest_browserandtest_browser_react_18and nothing else.maxWorkers: 2stays invitest.shared.mtsfor local runs and for the jsdom job, which shards with--no-file-parallelismand does not have this problem.This does not fix the underlying issue - pages of finished projects still stay open until the end of the run, which is what vitest#10991 addresses. It buys enough headroom in the meantime, without sharding, report merging, or waiting on an upstream release.
Confirmed on CI
The table above was measured on a macOS laptop, so the container was the real test. It agrees: the
test_browserjob now peaks under 75% of the container's RAM, where it previously ran at the ceiling. All 12 CircleCI jobs pass.One note for anyone reading the check history:
test_browser_react_18failed once on an earlier run, ondependencyArrows.EventTimelinePremium.test.tsx. That is the roaming act-warning flake on that job, not this change - three unrelated branches hit it on three different files within 40 minutes that morning, one of them going green on retry, and it passed here on a re-run of identical code.x-scheduler-premiumalso passes locally at bothVITEST_MAX_WORKERS=1and=2.