Run 192: trial of apply groups side by side (braid#306) #484

Closed
opened 2026-11-11 01:47:07 -05:00 by cmoriarty · 3 comments
Owner

What this is: run 192 applies run 189's brief side by side (parallel, at most 3 groups at once), from run 189's starting commit, on production fb12307. It is the trial for braid#306. Repository: braid-testing/scratch-parallel-190, thorough, auto-approve, vLLM qwen3.8-27b. Runs 190 and 191 were stopped within seconds: I put the setting back before they had kept it.

To measure against run 189 (660 min, apply groups 528 min):

  • How many groups the plan makes independent, and how many actually run at once.
  • Merge conflicts when a group's worktree merges back, and what they cost.
  • Apply and total time; generation speed per session under load.

Also checks braid#481 in a full run: browser tests run by the group that writes them, attempts lost to the now-blocking text-only rule, and the code review using helpers first.

Findings arrive below.

**What this is:** run 192 applies run 189's brief side by side (`parallel`, at most 3 groups at once), from run 189's starting commit, on production `fb12307`. It is the trial for braid#306. Repository: `braid-testing/scratch-parallel-190`, thorough, auto-approve, vLLM qwen3.8-27b. Runs 190 and 191 were stopped within seconds: I put the setting back before they had kept it. **To measure against run 189 (660 min, apply groups 528 min):** - How many groups the plan makes independent, and how many actually run at once. - Merge conflicts when a group's worktree merges back, and what they cost. - Apply and total time; generation speed per session under load. **Also checks braid#481 in a full run:** browser tests run by the group that writes them, attempts lost to the now-blocking text-only rule, and the code review using helpers first. **Findings** arrive below.
Author
Owner

Run 192: result

Succeeded in 403.5 min against run 189's 660, with 446 agent minutes against 659, merged with no fix left. Lanes went red once on the lint, the Visual check failed 1 of 27 once, and round 3 passed 27 of 27.

Run 189 (one group at a time) Run 192 (side by side)
Groups 9 7
Apply, start to end 527 min 283 min
Apply, agent minutes 528 333
Groups at once 1 2 for 49 min (groups 1 then 2 beside group 3), then 1
Merge conflicts none possible 1 of 7 merges (styles.css), resolved by the agent in 4.9 min
Tokens/s per agent 50 to 66 41 to 45 while two ran, 50 to 56 alone

Side by side saved about 35 to 40 minutes: groups 1 and 2 ran inside group 3's 94 minutes, and group 3 ran about 18% slower while they did. Two agents gave about 1.6 times the work of one. Most of the 257 minutes saved came from elsewhere: a plan with less work in it, and no group 7 spending two hours on browser tests nobody had run. After group 3, the plan chained groups 4 to 7, so nothing more could overlap.

braid#481 in a full run

Fix Seen
Browser tests run by their group yes: group 7 ran its tests 6 times; the planner still wrote --list (below)
B8 settings file yes: group 1 passed first time
Help files yes: group 6's retry and the fix round read theirs
Requirement limit yes: 9 requirements over 500 failed validation; repair took 5.6 min
Refusal inside the run's folder yes: gives the real reason
Explore's openspec/ line yes: gone
B11 blocks blocks, but on false positives (below)
Code review helpers never offered (below)
Bugs file cleaning, tested_by not exercised: no bug noted, no tested by entry

Problems, worst first

  1. The text-only rule (B11) flags tests that read no file, and it now fails the round. One test sets const html = readFileSync(index.html); two others set const html = renderRoom(...), and the rule treats every html in the file as file text. Two of round 1's three failures were false. The fix round moved them into the browser test (7.2 min, another lane round, a slower suite). Fix: judge the names a test reads within that test.
  2. The code review was never offered its helpers (bug since braid#474). code-review is missing from DELEGATION_WHOLE, so has_subagent is false and the "read the change through reader" line never renders. That explains runs 189 and 192 using none. The #474 test replaced has_subagent and hid it.
  3. Side by side, a compacted group cannot read its saved brief. brief_path uses the worktree's parent plus tmp, which for a group's worktree is groups/tmp. 11 reads were refused across groups 3, 5, 6 and 7. Group 3 then tried to run Braid's package to learn the hand-over format. Fix: the run's own tmp.
  4. The planner still writes --list for the task that writes the browser test. It read "NO task to run the whole suite" as overriding the new clause. Group 7 ran its tests anyway, because the apply brief says to.
  5. A test of wall-clock timing shipped. session.test.ts's "bot pacing" compares Date.now() differences, and it failed once under load in group 1. The lint catches fixed waits but not timing assertions.
  6. Group 6 lost 41 minutes to one hand-over entry. Its first attempt listed game-table/Self-hosted faces with "I looked" as the reason and failed at the end. The retry took 9 minutes, after reading the help file.
  7. A group started a server in the shell and stopped it with pkill -f, which matches any such process the agents' user owns: a sibling group's or another run's (braid#201). The fixed port 4173 would also collide between groups.
  8. The browser lane takes 2 minutes (full games at Fast), against 18 s in run 189. Each lane round pays it, and the fix round moved two unit tests into it.
  9. Minor: repair counted characters by hand; the limit's message could say what is counted (the text between the heading and the first scenario).

Runs 190 and 191 were stopped within seconds: I put the apply mode back before they had kept it.

## Run 192: result Succeeded in **403.5 min** against run 189's 660, with 446 agent minutes against 659, merged with no fix left. Lanes went red once on the lint, the Visual check failed 1 of 27 once, and round 3 passed 27 of 27. | | Run 189 (one group at a time) | Run 192 (side by side) | |---|---|---| | Groups | 9 | 7 | | Apply, start to end | 527 min | 283 min | | Apply, agent minutes | 528 | 333 | | Groups at once | 1 | 2 for 49 min (groups 1 then 2 beside group 3), then 1 | | Merge conflicts | none possible | 1 of 7 merges (`styles.css`), resolved by the agent in 4.9 min | | Tokens/s per agent | 50 to 66 | 41 to 45 while two ran, 50 to 56 alone | **Side by side saved about 35 to 40 minutes**: groups 1 and 2 ran inside group 3's 94 minutes, and group 3 ran about 18% slower while they did. Two agents gave about 1.6 times the work of one. Most of the 257 minutes saved came from elsewhere: a plan with less work in it, and no group 7 spending two hours on browser tests nobody had run. After group 3, the plan chained groups 4 to 7, so nothing more could overlap. ## braid#481 in a full run | Fix | Seen | |---|---| | Browser tests run by their group | **yes**: group 7 ran its tests 6 times; the planner still wrote `--list` (below) | | B8 settings file | **yes**: group 1 passed first time | | Help files | **yes**: group 6's retry and the fix round read theirs | | Requirement limit | **yes**: 9 requirements over 500 failed validation; repair took 5.6 min | | Refusal inside the run's folder | **yes**: gives the real reason | | Explore's `openspec/` line | **yes**: gone | | B11 blocks | blocks, **but on false positives** (below) | | Code review helpers | **never offered** (below) | | Bugs file cleaning, `tested_by` | not exercised: no bug noted, no `tested by` entry | ## Problems, worst first 1. **The text-only rule (B11) flags tests that read no file, and it now fails the round.** One test sets `const html = readFileSync(index.html)`; two others set `const html = renderRoom(...)`, and the rule treats every `html` in the file as file text. Two of round 1's three failures were false. The fix round moved them into the browser test (7.2 min, another lane round, a slower suite). Fix: judge the names a test reads within that test. 2. **The code review was never offered its helpers (bug since braid#474).** `code-review` is missing from `DELEGATION_WHOLE`, so `has_subagent` is false and the "read the change through `reader`" line never renders. That explains runs 189 and 192 using none. The #474 test replaced `has_subagent` and hid it. 3. **Side by side, a compacted group cannot read its saved brief.** `brief_path` uses the worktree's parent plus `tmp`, which for a group's worktree is `groups/tmp`. 11 reads were refused across groups 3, 5, 6 and 7. Group 3 then tried to run Braid's package to learn the hand-over format. Fix: the run's own `tmp`. 4. **The planner still writes `--list` for the task that writes the browser test.** It read "NO task to run the whole suite" as overriding the new clause. Group 7 ran its tests anyway, because the apply brief says to. 5. **A test of wall-clock timing shipped.** `session.test.ts`'s "bot pacing" compares `Date.now()` differences, and it failed once under load in group 1. The lint catches fixed waits but not timing assertions. 6. **Group 6 lost 41 minutes to one hand-over entry.** Its first attempt listed `game-table/Self-hosted faces` with "I looked" as the reason and failed at the end. The retry took 9 minutes, after reading the help file. 7. **A group started a server in the shell and stopped it with `pkill -f`**, which matches any such process the agents' user owns: a sibling group's or another run's (braid#201). The fixed port 4173 would also collide between groups. 8. **The browser lane takes 2 minutes** (full games at Fast), against 18 s in run 189. Each lane round pays it, and the fix round moved two unit tests into it. 9. Minor: repair counted characters by hand; the limit's message could say what is counted (the text between the heading and the first scenario). Runs 190 and 191 were stopped within seconds: I put the apply mode back before they had kept it.
Author
Owner

The owner went through these one at a time. The approved fixes are filed as braid#485. Left out: checking the hand-over files when written, and refusing pkill. More side-by-side work waits for isolated environments (braid#201, braid#203).

The owner went through these one at a time. The approved fixes are filed as braid#485. Left out: checking the hand-over files when written, and refusing pkill. More side-by-side work waits for isolated environments (braid#201, braid#203).
Author
Owner

Closing. Run 192 ended and the owner went through its findings (above). The approved fixes shipped as braid#485 and are on production (f27905f).

What is left of side by side stays on braid#306, paused until agents run in isolated environments (braid#203). The next full run measures the #485 fixes on its own findings issue.

Closing. Run 192 ended and the owner went through its findings (above). The approved fixes shipped as braid#485 and are on production (`f27905f`). What is left of side by side stays on braid#306, paused until agents run in isolated environments (braid#203). The next full run measures the #485 fixes on its own findings issue.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#484
No description provided.