What run 172 lost: forks with room, later rounds that look at what changed, browser and test fixes (#383 #378 #379 #374 #381 #382 #376 #384 #268 #386 #375 #373) #390

Merged
cmoriarty merged 12 commits from claude/batch-run-172 into main 2026-11-04 23:32:12 -05:00
Owner

Seven changes for twelve issues, all about time or quality lost on run 172 (503 minutes, against run 171's 105 on Free).

What changes

  • Forks and handoffs (#383, #378). A step forks a session only when its last reading is under half its alarm, and a failed fork tries the next name. A fresh session's note carries its subagents' reports. A judge writes its report stub first, and is asked once if it goes idle without the report. An alarm in a continued session names the fresh session.
  • Later Visual check rounds (#379, #374). A later round is told which files changed and what passed, rechecks what a changed file could affect, and carries the rest; a carry no earlier round passed fails the round. An id left to the check that names a requirement is judged, and one that names nothing fails the round.
  • Browser checks (#381, #382). Building steps run browser in the background. A check has 60 turns, and an attempt six checks across its sessions. Named screenshots work, browser_emulate_media is allowed, and a probe holds an animation frame.
  • Tests (#376, #384, #268). Briefs carry Braid's own standard. A scenario can go untested with a reason, which the code review and this description show. A tests-only step is refused a package install at the call, and its retry keeps what it read.
  • Briefs (#386). One verdict rule, fixloop check-report, a literal tag example, ticks on a green report, and fewer second reads.
  • Coding pane (#375). A message's words stay above the call they led to, and a pinned pane follows any change in height.
  • Free (#373). A refusal about load spends a model for two minutes and teaches nothing. A learned count lasts a week, and one saved with no time is dropped on load, which clears production's false limit.

Where measuring changed the plan

  • #386's move of the delta specs is dropped. The 81k the apply session started at was mostly inherited fork history: its fresh session started at 25.9k, brief included. The move would undo #170.
  • #382's 60 s cap is dropped. 14 page scripts ran past 60 s and returned a result.
  • #381 and #382 were smaller than filed, and #375's cause was different. See their archived designs.

Verification

  • ./tools/test.sh passes, and all 83 simulated runs pass.
  • Scripted replays of run 172's rounds 2 and 3: the ask fires where round 3 went idle.
  • Free replays of run 172's steps, on other models, so not like for like:
    • Visual check round 3: 14.5 to 35.3 minutes across seven replays, against 75 or more. The last, on the final brief: 19 of 19 in 20.1 minutes. One earlier replay failed its step's check for a reason it did not print, on a brief since changed.
    • Spec tests: 3.4 to 5.4 minutes and 2 to 4 tests, against 40 minutes and 14 tests.

Not proved here

Speed on vLLM, since trogdor is busy elsewhere, and whether carrying passed criteria ever hides a real regression. Watch that on the next full run. #268 stays open: its done-when needs a planned project of several changes on a real model.

🤖 Generated with Claude Code

Seven changes for twelve issues, all about time or quality lost on run 172 (503 minutes, against run 171's 105 on Free). ## What changes - **Forks and handoffs (#383, #378).** A step forks a session only when its last reading is under half its alarm, and a failed fork tries the next name. A fresh session's note carries its subagents' reports. A judge writes its report stub first, and is asked once if it goes idle without the report. An alarm in a continued session names the fresh session. - **Later Visual check rounds (#379, #374).** A later round is told which files changed and what passed, rechecks what a changed file could affect, and carries the rest; a carry no earlier round passed fails the round. An id left to the check that names a requirement is judged, and one that names nothing fails the round. - **Browser checks (#381, #382).** Building steps run `browser` in the background. A check has 60 turns, and an attempt six checks across its sessions. Named screenshots work, `browser_emulate_media` is allowed, and a probe holds an animation frame. - **Tests (#376, #384, #268).** Briefs carry Braid's own standard. A scenario can go untested with a reason, which the code review and this description show. A tests-only step is refused a package install at the call, and its retry keeps what it read. - **Briefs (#386).** One verdict rule, `fixloop check-report`, a literal tag example, ticks on a green report, and fewer second reads. - **Coding pane (#375).** A message's words stay above the call they led to, and a pinned pane follows any change in height. - **Free (#373).** A refusal about load spends a model for two minutes and teaches nothing. A learned count lasts a week, and one saved with no time is dropped on load, which clears production's false limit. ## Where measuring changed the plan - **#386's move of the delta specs is dropped.** The 81k the apply session started at was mostly inherited fork history: its fresh session started at 25.9k, brief included. The move would undo #170. - **#382's 60 s cap is dropped.** 14 page scripts ran past 60 s and returned a result. - **#381 and #382 were smaller than filed,** and **#375's cause was different.** See their archived designs. ## Verification - `./tools/test.sh` passes, and all 83 simulated runs pass. - Scripted replays of run 172's rounds 2 and 3: the ask fires where round 3 went idle. - Free replays of run 172's steps, on other models, so not like for like: - Visual check round 3: 14.5 to 35.3 minutes across seven replays, against 75 or more. The last, on the final brief: 19 of 19 in 20.1 minutes. One earlier replay failed its step's check for a reason it did not print, on a brief since changed. - Spec tests: 3.4 to 5.4 minutes and 2 to 4 tests, against 40 minutes and 14 tests. ## Not proved here Speed on vLLM, since trogdor is busy elsewhere, and whether carrying passed criteria ever hides a real regression. Watch that on the next full run. #268 stays open: its done-when needs a planned project of several changes on a real model. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
sessions-carry-evidence (#383 #378), later-rounds-look-at-what-changed (#379 #374),
browser-checks-that-fit (#381 #382), tests-for-a-risk (#376 #384 #268), brief-hygiene (#386),
pane-keeps-order (#375), load-refusals-teach-nothing (#373).

Measured on run 172: forks start at 139k to 179k of a 196k alarm and respawn within 8 to 27
requests, so the fork, not the brief, is what fills a later round's context. That drops #386's
move of the delta specs, which would undo #170. 14 page scripts ran past 60 s and returned a
result, which drops #382's 60 s cap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On run 172 forks started at 139k to 179k of a 196k alarm and moved to a fresh session within 8 to
27 requests. A step now forks a session only when its last request read under half the alarm, and a
fork that fails tries the next name, as the first fix round's 404 needed.

An alarm in a continued session names the fresh session, and nothing says compacting. Braid's note
carries the reports of the attempt's finished subagents and stops sending the new session to the
diff. A judge writes its stub before its first look, held by the guard, which now reads the step
from `.osf/step.json`, and a judge that goes idle without its report is asked once.

Three simulated runs change on purpose: `apply-single` and `apply-auto-plentiful` expect the new
alarm action on a forked apply, and `judge-unfinished` scripts the turn after the ask.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On run 172 each Visual check round took as long as the first while the faults went from 6 to 1,
because the brief said to judge everything afresh and never named what the fix round touched. A
later round is now given the changed files, from the two rounds' records, and what passed. It
rechecks what a changed file could affect and carries the rest, and a carry no earlier round passed
fails the round. It is pasted only the delta specs still open. Round 1 is unchanged.

On run 171 an id in `.osf/test/visual.json` named a requirement, and `with_visual` dropped it without
a word. Such an id now reaches the check, and one that names nothing fails the round. The proposal's
review is asked to flag a requirement whose scenarios would pass with it unbuilt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On run 172 apply sat idle through six foreground `browser` calls, all cut at 40 turns, and a spent
quota with no word to stop sent it off to write its own Playwright harness. A building step's brief
now says to start `browser` in the background. A check has 60 turns. The quota counts calls across
an attempt's sessions, six with one cut check given back, and its refusal says to stop verifying.

#382's "Playwright defects" were Braid's own. A named screenshot lands in the run's folder under
that name, `browser_emulate_media` is allowed so reduced motion can be judged, and a trailing
semicolon no longer breaks the evaluate wrapper. A shipped probe holds an animation frame in one
call. #382's 60 s cap is dropped: 14 page scripts on run 172 ran past it and returned a result.

`browser-checks-capped` and `browser-check-out-of-turns` change on purpose: six checks, not four,
with the stub written first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A spec-test step was driven by scenario coverage: a session for every scenario with no tag, whether
or not a test would protect anything. Every test-writing brief now carries Braid's own standard, and
a scenario can be left untested with a reason in `.osf/test/untested.json`, which the code review
and the pull request both show, so it can't be a silent way round the rule. The pull request names a
lane past its target of about a minute, since the code review runs before the first lanes.

Run 172's spec-test step installed jsdom at 12.6 minutes against its brief and was rejected at 22. A
manifest edit or an added package is now refused at the call, using the step file, and the retry
after a rejection continues the rejected session, the one exception to an attempt forking afresh.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Run 172's judge spent about 7k reasoning tokens on two sentences that seemed to disagree, and each
round wrote its own validator. The brief now states one rule, which is what `judge_failures` does,
and `fixloop check-report` runs a report through the step's own check. B6 shows where a tag goes, a
green `test` report means ticking in that turn, and reviewers are told what needs no second read.

#386's move of the delta specs to files is dropped: the same step's fresh session started at 25.9k,
brief included, so its 81k was mostly inherited history, and the move would undo #170.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
opencode logs a text or reasoning part once, when it ends, and a model's words end only when its
stream does, after its tool call has started: on all 484 of run 172's apply messages with both. A row
keeps its first event's place, so a command sat above the thought that decided to run it. The adapter
now settles a message's open words when a tool starts, so every reader sees them in order, and their
end adds nothing or revises the same row.

Following fired a frame after the tail's text changed, and a code block that grew after that left the
view short. A ResizeObserver on the list now follows any change in height while pinned.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
OpenRouter answered one request "High demand ... limited to 15 requests per minute", Braid had
counted one that minute, and it learned a limit of 1 a minute for all of OpenRouter, for good. A
refusal about load now spends that model for two minutes and teaches nothing. A learned count lasts a
week, and one saved with no time is dropped on load, which clears production's reading at the next
deploy with no hand edit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`fork-leaves-no-room` runs the half-alarm rule end to end, since a replay runs one step and so has no
session to fork. `apply-group-compacts` expects the new alarm action on a forked group, on purpose.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Free replays of run 172's round 3 took 14.5 to 35.3 minutes, against 75 or more, and showed three
faults in the first cut. One carried `B5-table` from the round that failed it, and `check-report` said
it passed, so the round failed after all: a fix round for a slip the judge could correct. The report
check now runs the carry check, and the brief says a failed criterion is never carried. Two played a
whole game after a CSS fix, since a first look's instruction to play stood beside the later round's
rule: a later round that knows what changed now has only that rule. A criterion still `unclear` under
`carried_from` read as a false claim, though only a pass is one, and the brief now names what the last
round could not settle, to judge again.

A replay prints what failed, since a Free replay's stack goes with its container.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Free replays of run 172's round 3 wrote `"carried_from": "round 2"` where the schema takes a number,
and one judge checked only that its file parsed, then failed its step's check on a report it had
called valid. A real run retries that and corrects its form, at the cost of a retry. The brief now
shows the number, and running `check-report` is a numbered step of ending. A replay prints why its
step failed, which it did not, so that replay's cause was not seen.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chore(openspec): archive the run 172 batch
All checks were successful
test / suite (pull_request) Successful in 46s
54655b7eea
sessions-carry-evidence, later-rounds-look-at-what-changed, browser-checks-that-fit, tests-for-a-risk,
brief-hygiene, pane-keeps-order and load-refusals-teach-nothing, with their verification results.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid!390
No description provided.