Free profile: a model whose streams stall or time out keeps being chosen, and the router cuts slow streams at 10 minutes #195

Closed
opened 2026-10-18 02:15:33 -04:00 by cmoriarty · 4 comments
Owner

Seen in run 76, the first production run on the Free profile (97242c8), in openspec.propose.design. Every turn went to the strong chain's head, NVIDIA's z-ai/glm-5.3. Opencode showed provider.unknown: provider stream ended before completion and retried with "The previous response was interrupted. Continue from where you left off…", four times and counting, and the console sat idle for minutes between tries.

The router's log (GoModel 0.1.103), 2026-10-18:

  • 05:52:01Z to 05:52:29Z, four streams, and 06:02:31Z one more: provider stream ended before completion: context deadline exceeded (Client.Timeout …). That is the router's own http.timeout: 600, a flat ten-minute cap per request. Braid's served rows for the design step show duration_ms 600006.
  • 06:10:16Z: [nvidia] provider_error: provider stream stalled: no data for 5m0s. That is stream_idle_timeout: 300. The served row shows 460053 ms.
  • Every served row has failovers: [].

Why it keeps happening

  1. A stream that has started has sent its 200, so GoModel cannot fail over. The error arrives mid-stream as stream_error.
  2. Braid learns from refused attempts (statuses, quota markers) but not from a 200 that ended incomplete. So the model stays at the head of its chain, and opencode's retry goes straight back to it.
  3. The ten-minute total cap cuts a slow stream that is still sending: a slow free model reasoning at length is cut even while making progress. Waits should follow progress, with a hard cap only as a backstop.

To do

  1. Read GoModel's stream_error attempts (provider stream stalled, context deadline exceeded, stream ended before completion) as a failure of that model. Put it behind the other models of its tiers for a cool-down, so the next try is served by the next model. Say so in the quota view and Settings, e.g. "stalled: 3 streams cut in 20 min, back at 06:40".
  2. Make the router's stream bounds follow progress: keep the no-data idle bound, raise or drop the flat per-request cap for streams that keep sending (and check what GoModel 0.1.103 allows).
  3. A simulated-run scenario: a chain head whose streams stall, and the step finishing on the next model.

Workaround used now: take NVIDIA out (OSF_FREE_OPT_IN cleared in the stack).

Seen in run 76, the first production run on the Free profile (97242c8), in `openspec.propose.design`. Every turn went to the strong chain's head, NVIDIA's `z-ai/glm-5.3`. Opencode showed `provider.unknown: provider stream ended before completion` and retried with "The previous response was interrupted. Continue from where you left off…", four times and counting, and the console sat idle for minutes between tries. **The router's log (GoModel 0.1.103), 2026-10-18:** - 05:52:01Z to 05:52:29Z, four streams, and 06:02:31Z one more: `provider stream ended before completion: context deadline exceeded (Client.Timeout …)`. That is the router's own `http.timeout: 600`, a flat ten-minute cap per request. Braid's served rows for the design step show `duration_ms` 600006. - 06:10:16Z: `[nvidia] provider_error: provider stream stalled: no data for 5m0s`. That is `stream_idle_timeout: 300`. The served row shows 460053 ms. - Every served row has `failovers: []`. **Why it keeps happening** 1. A stream that has started has sent its 200, so GoModel cannot fail over. The error arrives mid-stream as `stream_error`. 2. Braid learns from refused attempts (statuses, quota markers) but not from a 200 that ended incomplete. So the model stays at the head of its chain, and opencode's retry goes straight back to it. 3. The ten-minute total cap cuts a slow stream that is still sending: a slow free model reasoning at length is cut even while making progress. Waits should follow progress, with a hard cap only as a backstop. **To do** 1. Read GoModel's `stream_error` attempts (`provider stream stalled`, `context deadline exceeded`, `stream ended before completion`) as a failure of that model. Put it behind the other models of its tiers for a cool-down, so the next try is served by the next model. Say so in the quota view and Settings, e.g. "stalled: 3 streams cut in 20 min, back at 06:40". 2. Make the router's stream bounds follow progress: keep the no-data idle bound, raise or drop the flat per-request cap for streams that keep sending (and check what GoModel 0.1.103 allows). 3. A simulated-run scenario: a chain head whose streams stall, and the step finishing on the next model. Workaround used now: take NVIDIA out (`OSF_FREE_OPT_IN` cleared in the stack).
Author
Owner

Another stream fault of the same class, run 76, openspec.apply[7], 06:57:51Z: provider.invalid-output: OpenAI Chat received content after the finish reason. The error body is a stream chunk with a read tool call and finish_reason: "tool_calls", so a model sent a tool-call delta after its finish reason. It is most likely Gemini through GoModel's translation: the probes saw Gemini 3.5's stream break opencode's first turn the same way. Like a stalled or cut stream, it ends the turn with an error after a 200. It belongs in point 1: count it against the model (cool-down), and continue the step on the next model.

Another stream fault of the same class, run 76, `openspec.apply[7]`, 06:57:51Z: `provider.invalid-output: OpenAI Chat received content after the finish reason`. The error body is a stream chunk with a `read` tool call and `finish_reason: "tool_calls"`, so a model sent a tool-call delta after its finish reason. It is most likely Gemini through GoModel's translation: the probes saw Gemini 3.5's stream break opencode's first turn the same way. Like a stalled or cut stream, it ends the turn with an error after a 200. It belongs in point 1: count it against the model (cool-down), and continue the step on the next model.
Author
Owner

Another output defect of the same class, run 78 (c1228b4), openspec.apply[5], 14:15:57Z to 14:16:18Z: opencode retried three times on status 200 Provider finish_reason: malformed_function_call (Gemini's MALFORMED_FUNCTION_CALL finish reason, passed through GoModel), on the same model. Like invalid_output, a retry on the same model tends to repeat it. Count it against the model (cool-down) so the retry goes to the next model.

Another output defect of the same class, run 78 (c1228b4), `openspec.apply[5]`, 14:15:57Z to 14:16:18Z: opencode retried three times on status 200 `Provider finish_reason: malformed_function_call` (Gemini's MALFORMED_FUNCTION_CALL finish reason, passed through GoModel), on the same model. Like `invalid_output`, a retry on the same model tends to repeat it. Count it against the model (cool-down) so the retry goes to the next model.
Author
Owner

Shipped in ead9f7e (change free-steps-ride-out-refusals, with #196), plus c1228b4: a client's cancel is not a cut (750de96), and the router fails over on unrenderable requests (#200). Seen on production in run 78: Gemini 3.5 Flash-Lite's malformed stream put it behind for 30 min (osf.free.stalled, cause invalid_output); a busy Zen model stalled (cause busy) and the step moved back to the router, announced before Braid's abort. Not yet seen live: a cut at 10 minutes under the new 3,600 s cap. Still open from the comments: malformed_function_call counted against the model.

Left open: the OpenSpec change is not archived yet, and its tasks are not ticked against production. Deferred when the session wrapped up (2026-10-18).

Shipped in ead9f7e (change `free-steps-ride-out-refusals`, with #196), plus c1228b4: a client's cancel is not a cut (750de96), and the router fails over on unrenderable requests (#200). Seen on production in run 78: Gemini 3.5 Flash-Lite's malformed stream put it behind for 30 min (`osf.free.stalled`, cause `invalid_output`); a busy Zen model stalled (cause `busy`) and the step moved back to the router, announced before Braid's abort. Not yet seen live: a cut at 10 minutes under the new 3,600 s cap. Still open from the comments: `malformed_function_call` counted against the model. Left open: the OpenSpec change is not archived yet, and its tasks are not ticked against production. Deferred when the session wrapped up (2026-10-18).
Author
Owner

Archived in OpenSpec as free-steps-ride-out-refusals (5a953a0, with #196). Verified on production in run 78: malformed streams and busy Zen models put behind, a client's cancel no longer counted, the router failing over on unrenderable requests (#200). Not yet seen live: a 10-minute cut under the 3,600 s cap, and malformed_function_call, which still needs counting; both are in #205. Closing.

Archived in OpenSpec as `free-steps-ride-out-refusals` (5a953a0, with #196). Verified on production in run 78: malformed streams and busy Zen models put behind, a client's cancel no longer counted, the router failing over on unrenderable requests (#200). Not yet seen live: a 10-minute cut under the 3,600 s cap, and `malformed_function_call`, which still needs counting; both are in #205. Closing.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#195
No description provided.