Review and judge: one review step, browser checks finish, the judge measures the look (#353 #354 #329 #346 #320 #269 #341 #335) #356

Merged
cmoriarty merged 10 commits from claude/batch-a-run-steps-judge-d10116 into main 2026-11-01 03:20:58 -05:00
Owner

Eight issues from run 166's review and the open judge work, a commit each, then one archive commit.

  • #353 One review step. Round one is agent.review.combined on every backend; the four-lens fan-out and the one_session/many_sessions uses go (the names stay only for recorded graphs). The digest no longer orders by n_lenses or lets one reading's lens files refute or overrule each other, and the briefs no longer say reviewers agreed. The thin-review rule is unchanged. The per-lens reread sims go; the review sims read as one reading.
  • #354 Browser checks finish, chips say what happened. A browser check that ran to its 40 turns is given back once, so the judge can send one continuation; a sixth check is still refused (#338). The briefs count key presses as actions and split long keyboard flows. A multi-key action wasn't feasible: the server presses one key a call and its code-running tool stays out. Chips: pass · 16 of 19 · 3 unclear (amber), 3 fixed.
  • #329 Each browser_evaluate races a 180 s limit in the page; a promise that never settles fails that call with a reason.
  • #346 The browser subagent and a self-driving judge are told to click, wait and pause document.getAnimations() in one evaluate, then screenshot.
  • #320 A .js/.mjs/.ts file counts as UI under a folder that a root index.html loads scripts from.
  • #269 The judge judges each whole screen as a person who knows that kind of app would, and gets two probes (text contrast, canvas region contrast), tested in Chromium on run 154's and 155's planted faults.
  • #341 A profile without its own run settings reads the defaults, never Free's flat copy; the page shows them and a first save stores them.
  • #335 The MVP scope paragraph is added once.

Checks: ./tools/test.sh green after rebasing on #355; all 75 sims passed before the rebase (2 fixed after the first full pass), smoke 7/7 after; openspec validate --all --strict clean. New sim browser-check-out-of-turns fails without the guard change. The new tests for #320, #335 and #341 fail without their fixes.

🤖 Generated with Claude Code

Eight issues from run 166's review and the open judge work, a commit each, then one archive commit. - **#353 One review step.** Round one is `agent.review.combined` on every backend; the four-lens fan-out and the `one_session`/`many_sessions` uses go (the names stay only for recorded graphs). The digest no longer orders by `n_lenses` or lets one reading's lens files refute or overrule each other, and the briefs no longer say reviewers agreed. The thin-review rule is unchanged. The per-lens reread sims go; the review sims read as one reading. - **#354 Browser checks finish, chips say what happened.** A `browser` check that ran to its 40 turns is given back once, so the judge can send one continuation; a sixth check is still refused (#338). The briefs count key presses as actions and split long keyboard flows. A multi-key action wasn't feasible: the server presses one key a call and its code-running tool stays out. Chips: `pass · 16 of 19 · 3 unclear` (amber), `3 fixed`. - **#329** Each `browser_evaluate` races a 180 s limit in the page; a promise that never settles fails that call with a reason. - **#346** The browser subagent and a self-driving judge are told to click, wait and pause `document.getAnimations()` in one evaluate, then screenshot. - **#320** A `.js`/`.mjs`/`.ts` file counts as UI under a folder that a root `index.html` loads scripts from. - **#269** The judge judges each whole screen as a person who knows that kind of app would, and gets two probes (text contrast, canvas region contrast), tested in Chromium on run 154's and 155's planted faults. - **#341** A profile without its own run settings reads the defaults, never Free's flat copy; the page shows them and a first save stores them. - **#335** The MVP scope paragraph is added once. **Checks:** `./tools/test.sh` green after rebasing on #355; all 75 sims passed before the rebase (2 fixed after the first full pass), smoke 7/7 after; `openspec validate --all --strict` clean. New sim `browser-check-out-of-turns` fails without the guard change. The new tests for #320, #335 and #341 fail without their fixes. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Round one is agent.review.combined, with no when:, whatever the run's parallel
sessions. The digest no longer orders by n_lenses or lets one reading's lens
files refute or overrule each other, and the briefs no longer say reviewers
agreed. The thin-review rule is unchanged. one_session and many_sessions stay
only for recorded graphs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A browser check that ran to its 40 turns no longer counts toward a session's
four, once, so the judge can send one continuation; checks that finish and
disagree still stop at four (#338). The briefs count key presses as actions
and split long keyboard flows: the browser server presses one key a call, so
no multi-key action. The Visual check's chip reads 'pass · 16 of 19 · 3
unclear' in amber, and the code review's says what it fixed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The guard runs each browser_evaluate function against a 180 s limit in the
page, so a promise that never settles fails that call with a reason and the
session goes on, instead of the watchdog redoing the step from its entry
checkpoint. Functions and expressions, with or without an element, behave as
before (checked in Chromium the way Playwright's server evaluates them).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Run 160's 1.5 s animation was over before each screenshot: click, wait and
screenshot were three turns of about 10 s on fastllm. The browser subagent and
a judge that drives the browser itself are told to click, wait and pause
document.getAnimations() in one browser_evaluate, then take the screenshot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A .js/.mjs/.ts file counts as UI when an index.html at the repository's root
loads scripts from its folder, so a fix to a Vite game's src/render.js gets a
Visual check; a Node service with no root page keeps its src/ out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Seeded faults caught 1 of 3: the judge measured a mirrored control through the
app's state, but read text near-black on black and a paddle in the background
colour off the code's comments. It now judges each whole screen as someone who
knows the kind of app would, and is given two probes to run in one evaluate:
text under WCAG's contrast floor, and a canvas region's contrast. Both are
tested in Chromium on run 154's and 155's faults.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A profile saved before profiles kept run settings read the flat copy, which is
Free's: run 160 on fastllm ran with one session, 4 subagents and 179,520 per
agent. It now reads the defaults, the Settings page lists it with them, and a
first save stores them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chore: archive the batch's eight changes (#353 #354 #329 #346 #320 #269 #335 #341)
All checks were successful
test / suite (pull_request) Successful in 1m3s
94095e798a
The main specs' merged requirements are held under 500 characters: round one's
review is split from how one reading's findings are counted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid!356
No description provided.