The per-test time limit fails fast tests on a busy machine #393
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What happens
./tools/test.shgoes red on a busy machine with nothing wrong. On 2026-11-05 it failed 3 of 5 runs at a load of 5 to 13: 6 failures in 4 tests, allN s against its 1 s limit, none an assertion. CI went red the same way in 7 runs (231 to 252), before #299's tmpfs. None has since, but a busy runner could do it again.Why
The limit reads the wall clock. At a load of about 30,
test_switching_profiles_brings_back_each_ones_run_settingstook 4.2 s on the clock, and 3.7 s of that was waiting for a free core. Its own time was 0.6 s.Fix
Charge a test for its own time: the wall clock less what the OS reports as waiting for a core. A real wait still counts.
Done when
The suite stays green at a measured load of about 30, and a test made truly slow (a long loop, a long wait) still fails.
Found while verifying #392.
Fixed in #397, not merged yet, so this stays open until it is. The change is archived on that branch.
The rule: a test is charged its wall clock, less the time the OS reports it waited for a core, less 8 s per CPU-second its children used, and never less than the CPU they used.
Green under load (backend, all 454 tests):
Truly slow still fails: a 1.5 s loop,
Event().wait(2),asyncio.sleep(2), a child onsleep 5, a child with 4 s of work, and a child with 0.5 s of work then an 8 s wait all fail, quiet and at a load of about 30.Not met in full: the done-when says the suite stays green. The whole suite passed 5 of 6 loaded runs. The red one was a browser stall that also happens with nothing else running: #396.
Also found: every read of the settings re-parsed the built-in pipeline. That is why
test_switching_profiles_brings_back_each_ones_run_settingskept tripping. It is parsed once now.