A Free request can think for minutes and end with nothing #366
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What happens
A Free request can run for minutes and end with nothing. On the Scratch run (#365), Cloudflare's
gemma-4-26b-a4b-itthought for 300 s, sending about 50 KB of reasoning and no tool call or answer, then the stream was cut. That was about 5 of the review step's 6.5 minutes. Braid retried on Qwen and benched the model for 30 minutes, but the time was already gone. Nothing ends a request short of the router's 3,600 s cap.Research first
Fix
Decide from the research. Candidates: limit requests that only think, or move models that get cut down their chains.
Done when
A sim shows a request that only thinks is given up on by the chosen limit, a healthy slow request isn't, and a Free replay of a step that was cut finishes sooner.
Research, first pass. One cut so far, so the numbers below are thin. They grow as the Scratch run goes on.
Braid's own 300 s timer didn't cause it.
STREAM_IDLE_Sis also 300, which looked suspicious, but GoModel 0.1.103 (internal/llmclient/stream_idle.go):provider stream stalled: no data for …The router's log for this request says
provider stream ended before completion: unexpected EOF, so the connection closed from the far side. That points at Cloudflare or the path to it. Cloudflare's Workers AI limits page gives no time limit, so 300 s is still unconfirmed.The model was stuck, not slow. The lost reasoning is 47,796 characters (7,314 words), and no tool call came. It ends in a loop: "I'll proceed. Wait, I just realized I should check…", with the same review question repeated 9 times.
A limit looks separable here, on thin data. All requests on this run so far (137):
kilo/reflection/beam-freecloudflare/…/gemma-4-26b-a4b-itovh/Qwen3.5-397B-A17BOnly the cut request passed 120 s. None of this covers the build step's big writes yet, or earlier runs, whose events are gone. #195 records GLM-5.3 streams still sending at 600 s, so a limit on total time would be wrong. It has to be on thinking with no tool call or answer.
Where a limit could live is the hard part.
Still open
Sources: Cloudflare Workers AI limits, a forum report of Workers AI timeouts on a Gemma model.
Numbers from run 170's build step (36.5 min): no cuts. 26 Gemma requests totalled 199 s, 16 on Qwen 397B totalled 116 s, 61 on
beam-freetotalled 159 s. The 184 on Zen's DeepSeek have no recorded duration, so they can't help with this. The run failed before its longer steps (#368), so the review's cut is still the only one, and question 2 (thinking time before the first tool call) is still open.