Most of the GPU's batch capacity goes unused: run independent steps side by side #128
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Split off from #121 (finding 12 of the scratch-run analysis,
docs/design/measurements/2026-10-02-scratch-run-efficiency.md).On trogdor one stream decodes about 48 tok/s. Seven streams decode about 33 tok/s each, roughly 230 tok/s together. Yet 75% of the model turns in the scratch runs had no other stream running: after the spec fan-out and the four review lenses, the pipeline is one step after another, so the run waits on a single stream for most of its wall clock.
Ideas:
Related: #125 (verification steps that redo each other).
Most of the GPU's batch capacity goes unused: run independent steps side by side, and measure speculative decodingto Most of the GPU's batch capacity goes unused: run independent steps side by sideApply groups side by side, one of the ideas above, is now #172: worktree per group, a three-way merge back into the run, conflicts resolved by the group's own session. It follows #171, because two groups at once need a port each and only opencode 2 can give a session its own environment.
Re-scoped against today's pipelines and #170, and shipped as
side-by-side-steps: a guard and documentation, with no new parallelism.What runs side by side already. The scheduler admits every ready step at once. The built-ins already overlap wherever it is safe: the delta specs beside the design, and the four review lenses. After the review gate, every agent step either edits the one worktree or must see it unchanged, like the judge.
The issue's two ideas:
agent.test.unitno longer exists.Measured. On production's seven thorough runs, two or more agent steps ran at once for only 4–9% of agent time. Apply groups took 376 of run 62's 508 agent minutes.
What shipped. A
named-pipelinesrequirement, held by unit tests over every built-in: no two implementation steps can run at once, and no committing or pushing step can run beside one.The real speed-up is running independent apply groups side by side, each in a worktree of its own. That is #172, split off from #170. Its dependency half (
Depends on:lines intasks.md) shipped with #170. Closing this in favour of #172.