Browser tests share a server within a job, and one fails for it now and then #461
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What happens
The browser tests share state within a CI job. Playwright runs four workers; each worker starts one test server with one database and runs its tests against it in turn. A run an earlier test seeded stays in that database, and the console gives every run under way a tab.
ui/e2e/console.spec.ts:22, "runs open as tabs, [ moves between them and x closes one without touching its run", fails now and then for it: PR 435 (2026-11-08) and PR 460 (2026-11-09), on changes that do not touch it. In the failing job the tab list held five runs, the test's own two and three left by earlier tests on worker 3, and afterxthe closed run's tab was still there when the 10 s poll gave up. It passes on a rerun and 40 times in a row on the laptop, where the same tests do not share a worker in the same order. Each failure costs a rerun by hand and a merge that waits on it.The runner is not the cause. Each job starts a fresh container, and the host is at full speed (measured 2026-11-09: SHA-256 of 200 MB in 600 ms on Braid's VM, 3,939 ms on the runner in October).
Fix
Both, decided by the owner on 2026-11-09, who accepts about a second more per test:
console.spec.ts:22first.docs/testing.mdsays the browser tests are isolated per test, and the time budget is moved to match.Done when
console.spec.ts:22passes 50 times in a row with the workers CI uses../tools/test.shstill fits its budget, with the new figure written down.The run-tabs browser test fails in CI now and then (console.spec.ts:22)to Browser tests share a server within a job, and one fails for it now and thenShipped in PR 468 (main
ddfd8a93), archived asbatch-2026-11-09b(PR 469).What changed. Every browser test starts its own server and database and stops it after (
ui/e2e/fixtures.ts, theserverfixture per test, fortestandtestOnFree). The run-tabs journey asserts on its two runs alone. A serial pair inconsole.spec.tsseeds a run in one test and shows the next test on the same worker finds/api/runsempty and no tab.docs/testing.mdsays the browser tests are isolated per test and carries the figures.Evidence on the laptop.
console.spec.ts:22with the four workers CI uses: 50 of 50../tools/test.sh: green in 46 s; the browser step 29 s on the laptop with four workers (was 13 s before the change on the same laptop, so about a second a test, as expected). CI's browser step on PR 468 is within its 60 s budget.