Playwright tool defects the judge works around every round: ignored screenshot filename, no media emulation, scripts that hang 180 s, animation capture reinvented #382
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What happens
Across run 172's three judge rounds (247
evaluatecalls against 71 clicks):take_screenshotignoresfilenameand writes timestamped names, so the judge'sreadof the file it named failed 8 times (landing.png,waiting-bots.png, ...).emulateMedia, so "reduced motion" wasunclearin all three rounds and the brief's criterion could never be judged.(opts) =>form failed 6 times ("{(intermediate value)} is not a function", "reading 'ms'"), and two evaluates hung 180 s each ("the script did not settle in 180 s").Fix
filenameon screenshots, or return the path written so the judge reads the right file.emulateMedia(reduced motion, colour scheme) to the browser toolset, or drop reduced motion from what a judge is asked to verify.evaluateat 60 s and say in the tool description that the function takes no arguments.document.getAnimations()at a givencurrentTime, screenshot, resume; offer it in the judge brief instead of having each round invent one.Done when
A judge round reads the screenshot it took, judges reduced motion, and captures an animation frame in one call. ~12 + 35 min on run 172.
From the run 172 transcript review (Scratch on vLLM + MTP k=4, 2026-11-04; report in the owner's
~/repos/vLLM/reports/). Related: braid#378, braid#379.Shipped in PR #390, production
ddd4b6b.browser_emulate_mediais allowed, and a page script may end in;.Measured on run 172, against what this issue says:
braid_guards.jsdeletedfilenameon purpose,boundEvaluateadds the wrapper and the 180 s, andemulate_mediawas left off the tool list.;was 2 of the 14 scripts that failed.Evidence:
tests/agent/test_path_guard.py(failed first) andui/e2e/judge-probes.spec.ts. In Free replays of round 3, judges set reduced motion withbrowser_emulate_mediaand judged it, which run 172 could not. Those replays carried the animation criteria, so the probe rests on its browser test.