Browser specs: a fixture backend per Playwright worker, so eight workers run and no spec file runs serially #228

Closed
opened 2026-10-20 19:23:16 -04:00 by cmoriarty · 1 comment
Owner

Tier 2 of the test-suite speed-up (#227). The browser step takes 6:06 for 365 specs because every worker shares one fake osfd (ui/dev/fakeosfd.py on 8719): one Python process with one scripted stream, so 8 workers measured slower than 4 and flaky, and three spec files that restart the scenario (/_fake/scenario) run one file at a time in a serial project.

Proposal: start one fixture per worker, on its own port (Playwright sets TEST_PARALLEL_INDEX in every worker, so a shared e2e/fake.ts can derive the port from it and the 48 specs import their base address from there instead of each reading OSF_FAKE_URL). The servers start in a global setup, one per worker, and stop in the global teardown; webServer goes. Then the serial scenario project is unnecessary and the worker count is measured at 4, 6 and 8. Expected: 6 minutes to about 3. A worktree keeps its own ports through OSF_FAKE_URL as before.

Tier 2 of the test-suite speed-up (#227). The browser step takes 6:06 for 365 specs because every worker shares one fake osfd (ui/dev/fakeosfd.py on 8719): one Python process with one scripted stream, so 8 workers measured slower than 4 and flaky, and three spec files that restart the scenario (`/_fake/scenario`) run one file at a time in a serial project. Proposal: start one fixture per worker, on its own port (Playwright sets TEST_PARALLEL_INDEX in every worker, so a shared `e2e/fake.ts` can derive the port from it and the 48 specs import their base address from there instead of each reading OSF_FAKE_URL). The servers start in a global setup, one per worker, and stop in the global teardown; `webServer` goes. Then the serial `scenario` project is unnecessary and the worker count is measured at 4, 6 and 8. Expected: 6 minutes to about 3. A worktree keeps its own ports through OSF_FAKE_URL as before.
Author
Owner

Shipped (29eaee0). ui/e2e/global-setup.ts starts a fake osfd per worker on the base port plus the worker's index; ui/e2e/fake.ts gives each spec its address; the serial scenario project is gone; a fixture already answering on a port is reused, and OSF_FAKE_URL moves a worktree's base.

Measured alone, all 369 specs: before 366 s on 4 workers with the shared fixture; with a fixture per worker 241 s; 6 workers 290 s with a flake; 8 workers drove the load average to 87 and failed broadly. The next cost was video: every test paid an encoder for a recording kept only on failure. Without it, 4 workers took 150 s, all green, so video is off unless OSF_E2E_VIDEO=1 (the trace still keeps a failure's screenshots and actions). The worker count stays at Playwright's default.

Shipped (`29eaee0`). `ui/e2e/global-setup.ts` starts a fake osfd per worker on the base port plus the worker's index; `ui/e2e/fake.ts` gives each spec its address; the serial `scenario` project is gone; a fixture already answering on a port is reused, and `OSF_FAKE_URL` moves a worktree's base. Measured alone, all 369 specs: before 366 s on 4 workers with the shared fixture; with a fixture per worker 241 s; 6 workers 290 s with a flake; 8 workers drove the load average to 87 and failed broadly. The next cost was video: every test paid an encoder for a recording kept only on failure. Without it, 4 workers took **150 s**, all green, so video is off unless `OSF_E2E_VIDEO=1` (the trace still keeps a failure's screenshots and actions). The worker count stays at Playwright's default.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#228
No description provided.