A stopped run can't take a backend change: let it adopt its profile's new settings and re-plan apply on resume #352

Closed
opened 2026-10-31 15:47:55 -04:00 by cmoriarty · 1 comment
Owner

What happens

  • Since #305 and #313 a run keeps its own copy of its backend's settings from the moment it starts: endpoint, model, window, output reserve, slots, and the context per agent derived from them. Its opencode server is provisioned from that copy. Nothing can change the copy afterwards, even while the run is stopped.
  • On 2026-10-31, braid#350's run D (run 164) ran Flash-Next on stack 128, an ExLlamaV3 backend with a 131,072 window, so 110,592 per agent. Braid split its apply into six parts.
  • Mid-run, the owner raised the stack to a 262,144 window (8-bit KV cache, same VRAM). That is enough room for a single apply, since auto applies in one step at 200,000 or more per agent.
  • Run 164 was stopped cleanly, but it could not take the new window:
    • its settings copy still said 131,072 − 20,480;
    • its opencode.json had the old window;
    • the six-part apply plan was already fixed;
    • a step redo would only replay the same six parts.
  • The only way forward was a fresh run on the same brief. That threw away about 60 minutes of explore, propose, review and three apply parts.

What we want
A stopped (or failed) run can take a change to its backend and carry on with it, without breaking #313's guarantee that a live run isn't changed under its feet. This needs design, mainly around the profile versus the run's own copy. Some shapes:

  • Offer it when the profile changes: saving a profile that stopped or failed runs use asks "Also apply to these N runs?" and lists them. Live runs are never touched.
  • Or pull it from the run: a stopped run shows a "backend changed since this run started" notice with the difference (window 131,072 → 262,144, per agent 110,592 → 220,000) and a button to adopt it on resume. The run can also switch to a different profile.
  • Either way:
    • resuming re-provisions the run's opencode server from the updated copy;
    • the run's history records what changed and when.
  • The plan that depended on the old window: if no apply part has finished, the run can re-plan its apply split under the new context per agent. If parts have finished, it offers a choice between keeping the existing split and rewinding to before apply to re-plan, keeping the finished work in history.

Done when

  • Stopping a run, raising its profile's window, adopting the change and resuming continues the run at the new context per agent, and its opencode server reports the new window.
  • With no apply part finished, the resumed run re-plans the apply, giving one apply step at 200,000 or more per agent.
  • Live runs are unaffected by profile edits, as now.

Where to look:

  • the run's settings copy from #305 and #313;
  • POST /api/runs/{id}/stop and /resume;
  • opencode provisioning, provision() in src/osf/agent/server.py;
  • the apply split: .osf/budget.json#groups and BUDGET_GROUPS_FANOUT in src/osf/engine/executors.py, plus src/osf/engine/estimator.py;
  • POST /api/steps/{id}/redo and rerun-stale.

Related: #324, each run's own backend state in the top bar.

Found during braid#350 run D, 2026-10-31. Requested by the owner.

**What happens** - Since #305 and #313 a run keeps its own copy of its backend's settings from the moment it starts: endpoint, model, window, output reserve, slots, and the context per agent derived from them. Its opencode server is provisioned from that copy. Nothing can change the copy afterwards, even while the run is stopped. - On 2026-10-31, braid#350's run D (run 164) ran Flash-Next on stack 128, an ExLlamaV3 backend with a 131,072 window, so 110,592 per agent. Braid split its apply into six parts. - Mid-run, the owner raised the stack to a 262,144 window (8-bit KV cache, same VRAM). That is enough room for a single apply, since auto applies in one step at 200,000 or more per agent. - Run 164 was stopped cleanly, but it could not take the new window: - its settings copy still said 131,072 − 20,480; - its opencode.json had the old window; - the six-part apply plan was already fixed; - a step redo would only replay the same six parts. - The only way forward was a fresh run on the same brief. That threw away about 60 minutes of explore, propose, review and three apply parts. **What we want** A stopped (or failed) run can take a change to its backend and carry on with it, without breaking #313's guarantee that a live run isn't changed under its feet. This needs design, mainly around the profile versus the run's own copy. Some shapes: - **Offer it when the profile changes:** saving a profile that stopped or failed runs use asks "Also apply to these N runs?" and lists them. Live runs are never touched. - **Or pull it from the run:** a stopped run shows a "backend changed since this run started" notice with the difference (window 131,072 → 262,144, per agent 110,592 → 220,000) and a button to adopt it on resume. The run can also switch to a different profile. - **Either way:** - resuming re-provisions the run's opencode server from the updated copy; - the run's history records what changed and when. - **The plan that depended on the old window:** if no apply part has finished, the run can re-plan its apply split under the new context per agent. If parts have finished, it offers a choice between keeping the existing split and rewinding to before apply to re-plan, keeping the finished work in history. **Done when** - Stopping a run, raising its profile's window, adopting the change and resuming continues the run at the new context per agent, and its opencode server reports the new window. - With no apply part finished, the resumed run re-plans the apply, giving one apply step at 200,000 or more per agent. - Live runs are unaffected by profile edits, as now. **Where to look:** - the run's settings copy from #305 and #313; - `POST /api/runs/{id}/stop` and `/resume`; - opencode provisioning, `provision()` in `src/osf/agent/server.py`; - the apply split: `.osf/budget.json#groups` and `BUDGET_GROUPS_FANOUT` in `src/osf/engine/executors.py`, plus `src/osf/engine/estimator.py`; - `POST /api/steps/{id}/redo` and `rerun-stale`. Related: #324, each run's own backend state in the top bar. *Found during braid#350 run D, 2026-10-31. Requested by the owner.*
Author
Owner

Shipped in PR #421 (ff9381d8), change resume-adopts-profile, scoped to the second shape in the body. A stopped or failed run's detail carries backend_moved, the keys of its copy that differ from its profile as saved now (window, per agent, output reservation, effort, step levels, model, URL, …), and the run page shows "Resume with the backend as it is now" beside Resume (shift+R), with what moved in its hover. It rewrites the run's copy from the profile, records the adopted keys on osf.run.resumed, and the supervisor provisions the server from the new copy (a server still up from before the stop is replaced). Plain Resume keeps the copy; a live run is never changed. No re-plan step was needed: the apply fan-out decides one step or groups from the run's copy when it is reached, so a run stopped before apply gets one step on the larger window, and a run inside a split apply keeps its parts. Switching a run to another profile is not in this. Tests: tests/api/test_resume_adopt.py and a journey.

**Shipped in PR #421 (ff9381d8), change `resume-adopts-profile`, scoped to the second shape in the body.** A stopped or failed run's detail carries `backend_moved`, the keys of its copy that differ from its profile as saved now (window, per agent, output reservation, effort, step levels, model, URL, …), and the run page shows "Resume with the backend as it is now" beside Resume (`shift+R`), with what moved in its hover. It rewrites the run's copy from the profile, records the adopted keys on `osf.run.resumed`, and the supervisor provisions the server from the new copy (a server still up from before the stop is replaced). Plain Resume keeps the copy; a live run is never changed. No re-plan step was needed: the apply fan-out decides one step or groups from the run's copy when it is reached, so a run stopped before apply gets one step on the larger window, and a run inside a split apply keeps its parts. Switching a run to another profile is not in this. Tests: `tests/api/test_resume_adopt.py` and a journey.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#352
No description provided.