Skip to content

fix(ci): fix two flaky visual/browser test failures (firefox hang retry + Modal deflake) #2641

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
maaaathis merged 2 commits into from
Jul 15, 2026
+23 −4
Merged

fix(ci): fix two flaky visual/browser test failures (firefox hang retry + Modal deflake) #2641

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
bbd568c
ci(visual-tests): retry scheduled visual tests to absorb flaky firefo…
mfal Jul 14, 2026
338f054
test(Modal): wait for close animation instead of fixed sleep
mfal Jul 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions .github/workflows/test-visual-scheduled.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,24 @@ jobs:
run: pnpm test:browser:prepare --only-shell

- name: Run visual tests
# run without file parallelism, because firefox has issues with parallel visual tests
# Run without file parallelism, because firefox has issues with parallel visual tests.
# Retry the whole step because firefox intermittently *hangs* (not a snapshot diff) on a
# single visual test file in headless CI, which aborts the run with exit code 1 even though
# every test that runs passes. A hang can't be recovered by vitest's per-test `retry`, so we
# re-run the step with a fresh vitest/browser process. Successful `^build` deps are served
# from the nx cache, so retries only re-execute test:visual.
run: |
pnpm nx run-many -t test:visual --parallel=1 --browser.fileParallelism=false
for attempt in 1 2 3; do
echo "::group::Visual tests (attempt $attempt/3)"
if pnpm nx run-many -t test:visual --parallel=1 --browser.fileParallelism=false; then
echo "::endgroup::"
exit 0
fi
echo "::endgroup::"
echo "::warning::Visual tests failed on attempt $attempt/3 (likely a flaky firefox hang), retrying..."
done
echo "::error::Visual tests still failing after 3 attempts."
exit 1

- name: Send alerting message to Slack
if: failure()
Expand Down
8 changes: 6 additions & 2 deletions packages/components/src/components/Modal/Modal.browser.test.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -274,8 +274,12 @@ test("useOnClosed is called when the closing animation has finished", async () =
expect(modalText).toBeInTheDocument();
expect(onClosed).not.toHaveBeenCalledOnce();

await sleep(500);
expect(modalText).not.toBeInTheDocument();
// Wait for the close animation to finish and the Modal to unmount instead of
// relying on a fixed delay: the animation duration varies and a hard-coded
// sleep races against it under CI load, which made this test flaky.
await vitest.waitFor(() => expect(modalText).not.toBeInTheDocument(), {
timeout: 2000,
});

// onClosed is just called, when the Modal has unmounted (after close animation)
expect(onClosed).toHaveBeenCalledOnce();
Expand Down
Loading