Visual check rounds take as long as the first: later rounds are told to judge everything afresh and are not told what changed #379
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What happens
On run 172 (Scratch, thorough) each Visual check round took as long as the first, or longer, while the faults it was checking shrank:
The rounds are not fresh starts: round 2 forked round 1's session (45 messages inherited) and round 3 forked round 2's. But the judge behaves as if they were. Round 2 left one fault (the portrait reaction bar over the bid buttons), and round 3's subagents were "Full game, sounds, end, rematch", "Catch deal + trump flip animation", "Finish game, verify rematch restart", "Scoreboard + landing/waiting recheck" and five more.
Why
_fresh_look_lines(src/osf/engine/prompts.py) says: "Check each criterion that failed again, from a fresh page load, then go over the ones that passed, quickly." Three paragraphs earlier_recheck_linessays: "Check each again under the same id, and judge everything else afresh too: a fix can break what worked." The judge follows the stronger one.osf.budget.alarmat 13:16 and again in round 3), and a respawn throws the inherited memory away, so the fork buys nothing.fixloop.same_codecompares a digest to say whether the code changed since the last judge, but the brief never names the files, so the judge cannot tell which passed criteria a fix could have touched.B5-table) can only be rechecked by playing a game, so one such criterion forces a full playthrough every round.Fix (proposal), for rounds 2 and later
git diff --statbetween the two verify records).carried: truewith the round it was seen in._recheck_linesand keep the targeted wording in one place.Done when
A round that follows one with a single fault rechecks that fault and the criteria its fix could touch, carries the rest forward, and finishes in a fraction of the first round's time, with the first round unchanged.
Observed on run 172 (
run_01M6K8CA22F0FE8TNG6K6J6PG3), Visual check rounds 1-3, 2026-11-04; brief read at origin/maind53f9e9d. Related: braid#266, braid#282 (fresh browser per round), braid#165 (no-change claims), braid#378 (fork alarm wording).Shipped in PR #390, production
ddd4b6b.fixloop check-reportcatches it while the judge can still put it right.Evidence:
tests/pipeline/test_later_rounds.pyandtest_check_report.py. Seven Free replays of run 172's round 3 took 14.5 to 35.3 minutes, against 75 or more. They ran on other models, so not like for like.Not proved: whether carrying passed criteria ever hides a real regression. Watch that on the next full run.