Free profile: Kilo's beam-free refuses every session another model began, and 'behind' keeps sending it there #202

Closed
opened 2026-10-18 10:06:24 -04:00 by cmoriarty · 3 comments
Owner

Seen in run 78 (production c1228b4, the first run with #200's fix), openspec.apply[4], 2026-10-18 ~14:05Z:

  1. Gemini 3.5 Flash-Lite sent a malformed stream ("OpenAI Chat received content after the finish reason"). Braid put it behind for 30 minutes (osf.free.stalled, cause invalid_output), as #195 meant.
  2. The session's next request went to Kilo's reflection/beam-free: 400 "Missing required parameter: 'messages[44].reasoning_content'. This model needs the reasoning of every earlier assistant message: send back the 'reasoning_content' returned with that message."
  3. Braid saw a refused replay and moved the step to a fresh session from its own note (osf.budget.respawned, "the free models serving this step changed"), as #200 meant.
  4. The fresh session was served by OVH Qwen3.5 once, then by Gemini Flash-Lite six times. Gemini serves while marked stalled, because behind is not out and OVH's 2 requests a minute are soon used. Then Kilo got a request again: the same 400.

Beam can only continue a session whose every assistant message it wrote itself, or that carries reasoning_content. Any session another model touched is one it will refuse. #200 puts a refusing model behind the others for the attempt, never out, because the tier's virtual model is shared with the run's other steps. Behind still reaches Kilo whenever the models ahead are busy, so the refusal and the respawn repeat. The step makes progress between them, but each costs a respawn and a re-read.

To do (pick by design)

  1. Mark in the catalog a model that can only continue its own sessions (beam, from this refusal and the probes' replay matrix), and keep it off a session's chain once another family has served that session.
  2. Since the tier's virtual model is shared, consider a virtual model per step session (put by the router's admin API, as chains already are), so one session's chain can leave a model out without touching another step's.
  3. Or test whether an empty reasoning_content on foreign assistant messages satisfies beam's check (opencode or GoModel adding it), which would make the refusal go away.
Seen in run 78 (production c1228b4, the first run with #200's fix), `openspec.apply[4]`, 2026-10-18 ~14:05Z: 1. Gemini 3.5 Flash-Lite sent a malformed stream ("OpenAI Chat received content after the finish reason"). Braid put it behind for 30 minutes (`osf.free.stalled`, cause `invalid_output`), as #195 meant. 2. The session's next request went to Kilo's `reflection/beam-free`: 400 "Missing required parameter: 'messages[44].reasoning_content'. This model needs the reasoning of every earlier assistant message: send back the 'reasoning_content' returned with that message." 3. Braid saw a refused replay and moved the step to a fresh session from its own note (`osf.budget.respawned`, "the free models serving this step changed"), as #200 meant. 4. The fresh session was served by OVH Qwen3.5 once, then by Gemini Flash-Lite six times. Gemini serves while marked stalled, because behind is not out and OVH's 2 requests a minute are soon used. Then Kilo got a request again: the same 400. Beam can only continue a session whose every assistant message it wrote itself, or that carries `reasoning_content`. Any session another model touched is one it will refuse. #200 puts a refusing model **behind** the others for the attempt, never out, because the tier's virtual model is shared with the run's other steps. Behind still reaches Kilo whenever the models ahead are busy, so the refusal and the respawn repeat. The step makes progress between them, but each costs a respawn and a re-read. **To do (pick by design)** 1. Mark in the catalog a model that can only continue its own sessions (beam, from this refusal and the probes' replay matrix), and keep it off a session's chain once another family has served that session. 2. Since the tier's virtual model is shared, consider a virtual model per step session (put by the router's admin API, as chains already are), so one session's chain can leave a model out without touching another step's. 3. Or test whether an empty `reasoning_content` on foreign assistant messages satisfies beam's check (opencode or GoModel adding it), which would make the refusal go away.
Author
Owner

Deferred when the session wrapped up. An unfinished attempt is on the branch wip/202-beam-reasoning (d6c469c, not reviewed, lanes not run): a direct probe of Kilo, a fake router that refuses like beam unless every earlier assistant message carries non-empty reasoning, an opencode plugin braid_reasoning.js that adds it, and the sim scenario free-reasoning-required. The control check through GoModel and opencode reproduced the refusal; the check with the plugin against real Kilo was not finished. Run 78's apply[4], [5] and [6] each lost an attempt to this before succeeding (4, 6).

Deferred when the session wrapped up. An unfinished attempt is on the branch `wip/202-beam-reasoning` (d6c469c, not reviewed, lanes not run): a direct probe of Kilo, a fake router that refuses like beam unless every earlier assistant message carries non-empty reasoning, an opencode plugin `braid_reasoning.js` that adds it, and the sim scenario `free-reasoning-required`. The control check through GoModel and opencode reproduced the refusal; the check with the plugin against real Kilo was not finished. Run 78's apply[4], [5] and [6] each lost an attempt to this before succeeding (4, 6).
Author
Owner

Not seen on production yet. Shipped in #301 (5355515): the braid-reasoning plugin gives every earlier turn without reasoning a space. The simulated run free-reasoning-required fails with the plugin off (a respawn, as in run 78) and passes with it on.

Run 144 (Free, quick-fix) didn't reach the case. Beam served the implement session alone (19 requests, none refused), and in the summary's session Gemini Flash-Lite came after beam, not before it. This stays open until a Free run's call log shows beam continuing a session another model began. A scanner of a run's per-session call order is ready for the next Free run.

**Not seen on production yet.** Shipped in #301 (5355515): the `braid-reasoning` plugin gives every earlier turn without reasoning a space. The simulated run `free-reasoning-required` fails with the plugin off (a respawn, as in run 78) and passes with it on. Run 144 (Free, quick-fix) didn't reach the case. Beam served the implement session alone (19 requests, none refused), and in the summary's session Gemini Flash-Lite came after beam, not before it. This stays open until a Free run's call log shows beam continuing a session another model began. A scanner of a run's per-session call order is ready for the next Free run.
Author
Owner

Closed on the simulated run, by the operator's rule for #303 ("the sim can serve as proof"). Production has to happen to hand beam-free a session another model began, and no Free run has done that yet.

free-reasoning-required passed again today, in the full set of 72 on main (41f2cad). In it, beam-free serves the implementer's first turn, a 429 sends the second to OVHcloud's Qwen3.5, which writes no reasoning, and the third goes back to beam-free. The fake router refuses beam-free as Kilo does unless every earlier turn carries reasoning. With the braid-reasoning plugin, beam-free serves that turn in the same session: no refusal, no fresh session, and the step finishes. With the plugin off, the scenario fails with a respawn, as in run 78.

**Closed on the simulated run, by the operator's rule for #303** ("the sim can serve as proof"). Production has to happen to hand beam-free a session another model began, and no Free run has done that yet. `free-reasoning-required` passed again today, in the full set of 72 on main (41f2cad). In it, beam-free serves the implementer's first turn, a 429 sends the second to OVHcloud's Qwen3.5, which writes no reasoning, and the third goes back to beam-free. The fake router refuses beam-free as Kilo does unless every earlier turn carries reasoning. With the `braid-reasoning` plugin, beam-free serves that turn in the same session: no refusal, no fresh session, and the step finishes. With the plugin off, the scenario fails with a respawn, as in run 78.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#202
No description provided.