The apply leaves state-change scenarios to the Visual check because it looked at them itself #431

Closed
opened 2026-11-08 12:45:10 -05:00 by cmoriarty · 1 comment
Owner

What happens. The apply leaves scenarios whose outcome is a change of state to the Visual check, against the rule that only a scenario about appearance may be left to it (deliverable-judge, "Only a scenario about appearance is left to the Visual check"). Run 182's visual.json lists 28: 16 are appearance (the table, the card back, the animations, the scoreboard's look, reduced motion), and 11 are state changes with a testable outcome: create a room, join by code, join from the list, select then play, illegal cards disabled, collapse and expand, mute toggle, tap to play, keyboard play, help opens, help covers the rules. Each of the 11 gives the same reason, "browser-verified in task N": the apply looked at it in its own browser, counted that as verification, and wrote no test. Braid records nothing of the apply's own looks, so the Visual check did them all again. Tagged scenarios fell from 88 of 96 (run 174) to 44 of 77 (run 182), the Visual check's list grew from 17 to 34 criteria, and its clean time from 38 to about 78 minutes.

Fix. The implementer's own look is not a reason to leave a scenario to the Visual check. The apply brief says so, and says what is: the entry's reason names what a person sees that a test cannot assert. A check at script.scenarios rejects an entry whose reason cites the implementer's own check instead of an appearance, so the fix round writes the test. The Visual check is left only what needs a person's eyes, whatever the platform; no wording about browsers.

Done when. A brief test holds the sentence; a scenarios check test fails an entry reasoned "browser-verified in task 5.2" and passes one reasoned by what is seen; a scripted replay of run 182's apply leaves fewer state-change scenarios to the Visual check.

From run 182 (braid#425) and the wall-time analysis in docs/design/measurements/2026-11-08-runs-172-182-wall-time.md.

**What happens.** The apply leaves scenarios whose outcome is a change of state to the Visual check, against the rule that only a scenario about appearance may be left to it (deliverable-judge, "Only a scenario about appearance is left to the Visual check"). Run 182's `visual.json` lists 28: 16 are appearance (the table, the card back, the animations, the scoreboard's look, reduced motion), and 11 are state changes with a testable outcome: create a room, join by code, join from the list, select then play, illegal cards disabled, collapse and expand, mute toggle, tap to play, keyboard play, help opens, help covers the rules. Each of the 11 gives the same reason, "browser-verified in task N": the apply looked at it in its own browser, counted that as verification, and wrote no test. Braid records nothing of the apply's own looks, so the Visual check did them all again. Tagged scenarios fell from 88 of 96 (run 174) to 44 of 77 (run 182), the Visual check's list grew from 17 to 34 criteria, and its clean time from 38 to about 78 minutes. **Fix.** The implementer's own look is not a reason to leave a scenario to the Visual check. The apply brief says so, and says what is: the entry's reason names what a person sees that a test cannot assert. A check at `script.scenarios` rejects an entry whose reason cites the implementer's own check instead of an appearance, so the fix round writes the test. The Visual check is left only what needs a person's eyes, whatever the platform; no wording about browsers. **Done when.** A brief test holds the sentence; a scenarios check test fails an entry reasoned "browser-verified in task 5.2" and passes one reasoned by what is seen; a scripted replay of run 182's apply leaves fewer state-change scenarios to the Visual check. From run 182 (braid#425) and the wall-time analysis in `docs/design/measurements/2026-11-08-runs-172-182-wall-time.md`.
Author
Owner

Shipped in PR #437 (merged 6a8ee372), archived in PR #438 (visual-list-names-what-is-seen). A visual.json reason that cites the implementer's own check ("Browser-verified in task 5.6: …") and names no appearance counts for nothing: the scenario stays uncovered, the round's record lists it under visual_rejected, and "Where the scenarios stand" in the apply brief says having looked is not a reason, write the test or say what a person sees. The apply rules say the same. The look line no longer says to delegate by default (braid#425, finding 7).

Evidence: tests/pipeline/test_visual_scenarios.py splits run 182's own reasons as read by hand (the 16 appearance reasons kept, the 11 "browser-verified" ones rejected) and shows a rejected scenario staying uncovered; tests/engine/test_visual_list_brief.py holds the brief's sentences and the listing. ./tools/test.sh green. Not done: the replay of run 182's apply step; the next Scratch run counts what is left to the Visual check against run 182's 28.

**Shipped in PR #437 (merged 6a8ee372), archived in PR #438 (`visual-list-names-what-is-seen`).** A `visual.json` reason that cites the implementer's own check ("Browser-verified in task 5.6: …") and names no appearance counts for nothing: the scenario stays uncovered, the round's record lists it under `visual_rejected`, and "Where the scenarios stand" in the apply brief says having looked is not a reason, write the test or say what a person sees. The apply rules say the same. The look line no longer says to delegate by default (braid#425, finding 7). Evidence: `tests/pipeline/test_visual_scenarios.py` splits run 182's own reasons as read by hand (the 16 appearance reasons kept, the 11 "browser-verified" ones rejected) and shows a rejected scenario staying uncovered; `tests/engine/test_visual_list_brief.py` holds the brief's sentences and the listing. `./tools/test.sh` green. Not done: the replay of run 182's apply step; the next Scratch run counts what is left to the Visual check against run 182's 28.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#431
No description provided.