Planned projects run one item at a time: plan for the backend's concurrency #306
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What happens
A planned project starts its next item only after the last one merged and replanned. On vLLM, which serves six sessions at little cost, the Breakout project used one: its first item took 203 min (apply 66 min as one step, judge 29 min), and qwen3.8-27b's long thinking at ~60 tok/s is the bottleneck. The parallel review lenses bought little (#279: 1,240 s for four against 1,120 s for one combined reviewer).
Fix (to design)
Done when
The Breakout brief as a parallel project finishes in less total time than run 139's sequential baseline, with judge results no worse.
The operator's direction, 2026-10-29. Baseline: run 139.
Measured 2026-10-29: trogdor (vLLM, qwen3.8-27b) scales with sessions up to its cap of 6. N streams at once, 512 tokens each, short prompts:
So four sessions give 3.4x the work at 0.84x the speed each.
Prefill is the limit. Four sessions that each start with ~5,000 words of fresh context got their first token after up to 47 s, against 12 s alone. Agent turns mostly reuse cached context, so steady work should scale like the table. Fresh sessions and compactions contend.
Run 147, the Breakout project's item: its five
agent.write-spec-testssteps ran one after another, 1,905 s in all, with 5 of 6 slots free. The longest took 832 s. Side by side, each at ~0.8x speed, that is ~17 min instead of 32.Run 192 (braid#484) tried this: side by side saved about 35 to 40 of 403 minutes, because only two groups overlapped, each at about 0.8 times its speed alone. More gain needs the planner to make groups independent. Paused by the owner until agents run in isolated environments (braid#201, braid#203).
A fact that has changed since this was written: the vLLM server keeps two large contexts in its cache, not six (496,770 tokens on 2026-11-11). Braid now starts at most two agent steps at once on a local server (braid#498), and a step's helpers take places too (braid#502, braid#509). Three side by side made no more tokens a minute than one (braid#497, finding 17); two made about one and a half times (braid#505). Any plan for several items at once starts from two. The gaps left in the cap are braid#535.
Closing: the first slice shipped, and the rest is tracked elsewhere.
f63d2ca2, changeside-by-side-spec-tests, archived now).