A respawn or compaction hands over tool names, not evidence: judges redo passed checks, and a judge that went idle without its report loses 27 min #383

Closed
opened 2026-11-04 17:31:20 -05:00 by cmoriarty · 1 comment
Owner

What happens
Run 172 had five context alarms (apply 08:12 and 08:56, visual-check.r2, fix.r2, visual-check.r3). Braid's handoff note names the files and tool calls but not the subagent reports or the criteria already judged, so:

  • the respawned r2 judge redid "table animations/sweep/sound" (~10 min); the r3 judge redid portrait and deal/flip with three more subagents (~12 min); fix.r2, forked from the r1 fix session at ~178k input, respawned after 9 min and re-verified an already verified CSS fix (~6 min);
  • the r3 rematch subagent's report of a half-reset end screen ("R3 takes the game", 0 scratches, old scores still shown) was lost in the respawn that minute ("I don't have their reports") and the round passed it as "a miscount";
  • after the apply respawn no edit happened for 15 min: 13 full-file reads of its own code, two whole-suite runs, 38k reasoning tokens re-deriving the server; after the compaction all 14 client files just written were re-read (store.ts read 7x, App.tsx 4x over the run).
    Separately, r3 attempt 1 said "Every screen verified. Writing the round-3 report." and went idle without the write; the predicate failed it (judge.json is missing or empty, 14:48:58) and attempt 2 re-judged from scratch: 27 min lost. Neither r3 attempt wrote the finished: false stub the brief asks for, so the respawn had nothing to carry.

Fix

  • The handoff note lists the files the step wrote and whether their tests were green, keeps the last two or three tool results, and for a judge carries each subagent's final report: append every subagent report to .osf/fix/rN/checks/ as it arrives, so a respawn, a fork and the next round all read them.
  • Enforce the stub: refuse the first subagent call of a judge until judge.json exists with finished: false.
  • When a judge goes idle without the report, send "write it now" into the same session once before failing the attempt.
  • Fork a fix round from a fresh session with the repair note, not from the previous fix session's full context.

Done when
A respawned judge does not re-run a check its subagents already reported, a lost report cannot pass a round, and a judge that forgets the write is nudged rather than discarded. ~50 + 27 min on run 172.

From the run 172 transcript review (Scratch on vLLM + MTP k=4, 2026-11-04; report in the owner's ~/repos/vLLM/reports/). Related: braid#378, braid#379.

**What happens** Run 172 had five context alarms (apply 08:12 and 08:56, visual-check.r2, fix.r2, visual-check.r3). Braid's handoff note names the files and tool calls but not the subagent reports or the criteria already judged, so: - the respawned r2 judge redid "table animations/sweep/sound" (~10 min); the r3 judge redid portrait and deal/flip with three more subagents (~12 min); fix.r2, forked from the r1 fix session at ~178k input, respawned after 9 min and re-verified an already verified CSS fix (~6 min); - the r3 rematch subagent's report of a half-reset end screen ("R3 takes the game", 0 scratches, old scores still shown) was lost in the respawn that minute ("I don't have their reports") and the round passed it as "a miscount"; - after the apply respawn no edit happened for 15 min: 13 full-file reads of its own code, two whole-suite runs, 38k reasoning tokens re-deriving the server; after the compaction all 14 client files just written were re-read (`store.ts` read 7x, `App.tsx` 4x over the run). Separately, r3 attempt 1 said "Every screen verified. Writing the round-3 report." and went idle without the write; the predicate failed it (`judge.json is missing or empty`, 14:48:58) and attempt 2 re-judged from scratch: 27 min lost. Neither r3 attempt wrote the `finished: false` stub the brief asks for, so the respawn had nothing to carry. **Fix** - The handoff note lists the files the step wrote and whether their tests were green, keeps the last two or three tool results, and for a judge carries each subagent's final report: append every subagent report to `.osf/fix/rN/checks/` as it arrives, so a respawn, a fork and the next round all read them. - Enforce the stub: refuse the first `subagent` call of a judge until `judge.json` exists with `finished: false`. - When a judge goes idle without the report, send "write it now" into the same session once before failing the attempt. - Fork a fix round from a fresh session with the repair note, not from the previous fix session's full context. **Done when** A respawned judge does not re-run a check its subagents already reported, a lost report cannot pass a round, and a judge that forgets the write is nudged rather than discarded. ~50 + 27 min on run 172. _From the run 172 transcript review (Scratch on vLLM + MTP k=4, 2026-11-04; report in the owner's `~/repos/vLLM/reports/`). Related: braid#378, braid#379._
Author
Owner

Shipped in PR #390, production ddd4b6b.

  • A step forks a session only when its last reading is under half the alarm. A full session goes fresh, and a failed fork tries the next name.
  • A fresh session's note carries its subagents' reports.
  • A judge writes its report stub first, and is asked once if it goes idle without the report.

Measured on run 172: forks started at 81.7k (apply), 159.6k (round 2), 178.7k (second fix round), and 138.7k and 142.2k (round 3), against a 196k alarm. A fresh session starts at about 24k to 26k, brief included. The first fix round's fork got a 404 on a session whose worktree had gone.

Evidence: tests/engine/test_fork_room.py (failed first), test_note_reports.py, test_ask_for_report.py, and sims fork-leaves-no-room and judge-forgets-report. Scripted replays of rounds 2 and 3: the ask fires where round 3 went idle.

Not proved: speed on vLLM, since trogdor was busy.

**Shipped** in PR #390, production ddd4b6b. - A step forks a session only when its last reading is under half the alarm. A full session goes fresh, and a failed fork tries the next name. - A fresh session's note carries its subagents' reports. - A judge writes its report stub first, and is asked once if it goes idle without the report. **Measured on run 172:** forks started at 81.7k (apply), 159.6k (round 2), 178.7k (second fix round), and 138.7k and 142.2k (round 3), against a 196k alarm. A fresh session starts at about 24k to 26k, brief included. The first fix round's fork got a 404 on a session whose worktree had gone. **Evidence:** `tests/engine/test_fork_room.py` (failed first), `test_note_reports.py`, `test_ask_for_report.py`, and sims `fork-leaves-no-room` and `judge-forgets-report`. Scripted replays of rounds 2 and 3: the ask fires where round 3 went idle. **Not proved:** speed on vLLM, since trogdor was busy.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#383
No description provided.