Braid's decode speed counts the turn's tool time: a turn that runs tests or a subagent reads as a slow decode #344
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What happens
Braid times a request from when it was sent to opencode's
step-finish, butstep-finisharrives only after the turn's tools have run. So test runs and subagents count as decoding. On vLLM runs 139, 147, 152 and 156, 516 min of tool time was counted that way: the decode Braid recorded is 32.6 tok/s, where the model generated at 50.6. On run 160 (fastllm), the turn that started a 5-minute browser subagent reads as 1,485 tokens in 326 s (4.6 tok/s). The same turns also look like concurrent streams: 1,189 of those 1,875 vLLM requests seem to overlap another, but only 130 do. The Inference page's decode tile and/metricsgauges read low whenever a step runs tests or a subagent.Fix
End a request when its generation ends, not at
step-finish: at its last streamed token, or when its last tool call starts running (osf/throughput.py,RequestClock; the firehose already sees both).Done when
A turn that runs a 60-second command records its decode time without those 60 s, and a test covers a turn that calls a subagent.
Done in batch B, merged as #355 (
91f4b85f), and live on production: Braid reports commitc00c62f, which includes it. It was checked with the repo's tests and sims before the merge. A live check on vLLM is still to come: trogdor is off the network. Closed at the owner's request.