Re-run the pinball comparison after these fixes #260
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
So far (same brief, Claude Sonnet 5.5, time not spent waiting on a person)
Next, once the fixes land
soloand onminimalist, to see whatthorough's extra steps buy.claude -p --model sonnet, no CLAUDE.md). Last time it saw Braid's CLAUDE.md.Compare
Time, tokens, and a hands-on play of each game: look, physics, bugs.
Waits on: #244, #245, #246, #247, #248, #249, #250, #251, #252, #253, #254, #255, #256, #257, #258, #259.
From the Braid vs Claude Code pinball test, 2026-10-27 (runs 110–120).
Baseline from the first test (2026-10-27, Claude Sonnet 5.5 on both sides)
Braid one-shot is run 120. Braid planned is runs 110–116 for time (the same scope as Claude Code's game), and runs 110–119 for the rest.
Time (minutes)
Where Braid's time went (minutes, gate waits left out)
Tokens
Code and tests (each repo's main branch)
The one-shot kept its tests lean. The planned project's bloat came with one browser spec per change.
/**comment #264Findings from the re-run (final, after a full read of the transcripts of runs 121–132 and the hands-on play)
Bugs, filed
/**comment: 15 times in 9 of 12 runs; in run 125 it forced a step to run again.solonever asked its new web game forwindow.__app, and the judge's "unclear" play-through passed (run 122, the table with no drain).Smaller, not filed
tsc(58 runs, ~10 s each);tsc --incrementalwould cut most of that.check-replan's pass message ("PLAN.md keeps the plan's rules…") reads like a problem; agents second-guessed it four times.window.__app.statewithout calling it (3 times), and a few judge browser calls lost their page.Checked and fine
Now working as intended
/**comment #264Re-run after the fixes (#261, live as
5598ddf). Same brief, Claude Sonnet 5.5, auto-approve on, Use CI off (the new default). The three Braid runs ran side by side. Claude Code was not re-run.The same scope as Claude Code's game (minutes, not counting waits on a person)
thorough, run 120)solo, run 122) · 37.5 (minimalist, run 123)minimalistis slow for one reason: 8 judge rounds over the ball's look (#266).The whole planned project
Where the time went now (minutes)
solominimalistTokens
solominimalistCode and tests (each repo's main branch, all counted the same way)
solominimalistWhat changed
What didn't (bugs filed: see the findings comment)
Still to do
A hands-on play of each game:
cmoriarty/pinball-braid-2(planned),cmoriarty/pinball-braid-solo,cmoriarty/pinball-braid-minimalist.Hands-on play (the person's verdict)
soloturned the glow up too high: glaring and fuzzy.sololeft no drain between the flippers, so it's hard to lose a ball.minimalisthas badly set-up flippers; the right one faces the wrong way.The judge passed all three, so it missed each of these: #269.
That completes the re-run: results above, findings in the list.
solonever asks a new web project forwindow.__app, and an unclear play-through passes #272cmoriarty referenced this issue2026-11-01 02:15:50 -05:00