The Visual check can't see an animation shorter than a model turn: its screenshots land after it ends #346
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What happens
Run 160's brief asks for an "exciting" Hello World animation, which lasts 1.5 s. The Visual check's browser subagent clicked, waited and took the screenshot in separate turns. On fastllm each turn took about 10 s, so every screenshot showed the settled text. Its report: "I could not capture the mid-animation (coloured/scaled) state because the browser tool's own latency exceeded the 1.5 s animation duration". It judged the animation from the app's state hook without seeing it. Apply's own browser check hit the same thing, and worked around it by restarting the animation inside one
evaluateand pausing it. On vLLM a turn takes 1–3 s, still longer than most UI animations.Fix
Tell the browser subagent how to see motion: act and capture in one call. For example, one
evaluatethat clicks, waits N ms and pausesdocument.getAnimations(), then a screenshot. Or give it a tool that clicks and takes a screenshot after a delay.Done when
On a brief with a short animation, the Visual check's screenshots show the animation mid-flight, on vLLM and on fastllm.
Shipped in PR #356 (
c00c62f1, live on production).browser_evaluate: click, wait N ms, thendocument.getAnimations().forEach(a => a.pause()), and only then take the screenshot. For a canvas orrequestAnimationFrameloop, read the app's state in the same call.tests/engine/test_see_motion.pychecks that both briefs say it.Done in batch A, merged as #356 (
c00c62f1), and live on production: Braid reports commitc00c62f, which includes it. It was checked with the repo's tests and sims before the merge. A live check on vLLM is still to come: trogdor is off the network. Closed at the owner's request.