Repair, first real use (run 93): no way to stop it, a message waits behind a two-hour turn, its text does not stream, and it overreached #225

Closed
opened 2026-10-20 12:00:17 -04:00 by cmoriarty · 1 comment
Owner

Feedback from the first use of Repair (#222) on run 93, which failed at agent.write-spec-tests[ball-visualization]. I asked "Can you find out why this step failed?" and then, an hour later, "what is the current status?".

What the log shows

Repair's one turn ran from 22:28 to 00:29 (two hours): 99 model steps, 354 tool calls (192 shell, 81 edit, 27 write, 42 read). It found the cause early (its fourth text, after 11 s: the step wrote scratch-orient.ts, deleted it, and the tests-only predicate had seen it) and then, unasked, went on to run the new e2e specs, found real bugs in them, and began rewriting two test files. It was working, not stuck; it just never stopped. Its input reached 316,275 tokens, above the 150,000 context per agent every other step is held to: Repair's session has no context budget, compaction, deadline or last call.

What to change

  1. The placeholder. "Repair fixes what stopped this run and carries it on…" assumes the operator knows "Repair" is an agent. Replace it with an example of what someone would type, such as "Why did this step fail?" (and for a stopped step, one that fits two lines on a phone).
  2. Its text must stream, and it must show it is thinking. The console's live-text stream and its thinking indicator are keyed by step, and Repair's session belongs to none, so its turns appear only as each part settles, all at once, with no animation. Key the stream (and the "working" state) by the Repair session.
  3. It did far more than asked. Its instructions say to clear what stands in the way and carry the run on, so a request to find out why became a rewrite. It should do what the operator asked: a question gets an answer and a stop, and it should ask before changing files the question did not mention. Give its turns the bounds an agent step has (a context ceiling with compaction, a time limit, a last call).
  4. A message sent while Repair is working waits for the turn to end. The box sends with queue delivery, which lands after the current turn: ten minutes in the picture, and up to two hours here. It needs the step composer's controls: queue, redirect (stop the turn and send) and stop, shown while Repair is working. The chip should also say what is happening ("waiting for Repair to finish its turn"), not "it starts its server first" for ever.
  5. There is no way to stop it. The box drops Stop while it talks to Repair. Add Stop (an interrupt route for the run's Repair session, as steps have), and show whether Repair is working.

Where

ui/src/components/Composer.tsx, ui/src/lib/repair.ts, the delta stream in src/osf/agent/stream.py and ui/src/state/store.ts, src/osf/engine/repair.py (the prompt, the delivery mode, a route to interrupt), the budget code the agent executor uses for steps.

Follows #222; separate from #223 (Ask).

Feedback from the first use of Repair (#222) on run 93, which failed at `agent.write-spec-tests[ball-visualization]`. I asked "Can you find out why this step failed?" and then, an hour later, "what is the current status?". ## What the log shows Repair's one turn ran from 22:28 to 00:29 (two hours): 99 model steps, 354 tool calls (192 shell, 81 edit, 27 write, 42 read). It found the cause early (its fourth text, after 11 s: the step wrote `scratch-orient.ts`, deleted it, and the tests-only predicate had seen it) and then, unasked, went on to run the new e2e specs, found real bugs in them, and began rewriting two test files. It was working, not stuck; it just never stopped. Its input reached 316,275 tokens, above the 150,000 context per agent every other step is held to: Repair's session has no context budget, compaction, deadline or last call. ## What to change 1. **The placeholder.** "Repair fixes what stopped this run and carries it on…" assumes the operator knows "Repair" is an agent. Replace it with an example of what someone would type, such as "Why did this step fail?" (and for a stopped step, one that fits two lines on a phone). 2. **Its text must stream, and it must show it is thinking.** The console's live-text stream and its thinking indicator are keyed by step, and Repair's session belongs to none, so its turns appear only as each part settles, all at once, with no animation. Key the stream (and the "working" state) by the Repair session. 3. **It did far more than asked.** Its instructions say to clear what stands in the way and carry the run on, so a request to find out why became a rewrite. It should do what the operator asked: a question gets an answer and a stop, and it should ask before changing files the question did not mention. Give its turns the bounds an agent step has (a context ceiling with compaction, a time limit, a last call). 4. **A message sent while Repair is working waits for the turn to end.** The box sends with queue delivery, which lands after the current turn: ten minutes in the picture, and up to two hours here. It needs the step composer's controls: queue, redirect (stop the turn and send) and stop, shown while Repair is working. The chip should also say what is happening ("waiting for Repair to finish its turn"), not "it starts its server first" for ever. 5. **There is no way to stop it.** The box drops Stop while it talks to Repair. Add Stop (an interrupt route for the run's Repair session, as steps have), and show whether Repair is working. ## Where `ui/src/components/Composer.tsx`, `ui/src/lib/repair.ts`, the delta stream in `src/osf/agent/stream.py` and `ui/src/state/store.ts`, `src/osf/engine/repair.py` (the prompt, the delivery mode, a route to interrupt), the budget code the agent executor uses for steps. Follows #222; separate from #223 (Ask).
Author
Owner

Shipped to main in be3a6a4 (change repair-usability, archived). All five items from run 93:

  1. Placeholder is now an example prompt, Try: "Why did this step fail?"; for a stopped step it names Resume run and Redo and suggests asking why it stopped. Both fit two lines at 375px.
  2. Streaming: Repair's session has no step, so its deltas were dropped. They now go out under repair:<run id>, and the console subscribes while a Repair session is open. Text and thinking flow like a step's, with a working line until the first words.
  3. Overreach: Repair's instructions now say a question gets an investigation, a report and a stop; it changes files only when asked to fix something. A turn is also bounded: Settings, Repair's turn limit (30 minutes by default, 1 to 240), and the run's context ceiling. A stop at either is recorded (osf.repair.limit) and said in the conversation; after a context stop the next message begins a fresh session primed from the record.
  4. Queued follow-up: Queue still lands after the turn, but its chip now says it is waiting for Repair to finish its turn, and Redirect (stop, wait for idle, send) is offered.
  5. Stop: while Repair works the box shows Stop (Esc), Queue and Redirect, backed by POST /api/runs/{id}/repair/interrupt.

Verified with the full lane (4,724 backend tests, 942 vitest, 365 browser specs, 67 simulated runs) and a live test against the pinned opencode that stops a Repair turn and sends the next message. Not yet deployed: the head commit carries [skip ci], so production needs a manual deploy.

The apply question that proceeded with two questions unanswered is separate: braid#226.

Shipped to main in be3a6a4 (change `repair-usability`, archived). All five items from run 93: 1. **Placeholder** is now an example prompt, `Try: "Why did this step fail?"`; for a stopped step it names Resume run and Redo and suggests asking why it stopped. Both fit two lines at 375px. 2. **Streaming**: Repair's session has no step, so its deltas were dropped. They now go out under `repair:<run id>`, and the console subscribes while a Repair session is open. Text and thinking flow like a step's, with a working line until the first words. 3. **Overreach**: Repair's instructions now say a question gets an investigation, a report and a stop; it changes files only when asked to fix something. A turn is also bounded: Settings, Repair's turn limit (30 minutes by default, 1 to 240), and the run's context ceiling. A stop at either is recorded (`osf.repair.limit`) and said in the conversation; after a context stop the next message begins a fresh session primed from the record. 4. **Queued follow-up**: Queue still lands after the turn, but its chip now says it is waiting for Repair to finish its turn, and Redirect (stop, wait for idle, send) is offered. 5. **Stop**: while Repair works the box shows Stop (Esc), Queue and Redirect, backed by `POST /api/runs/{id}/repair/interrupt`. Verified with the full lane (4,724 backend tests, 942 vitest, 365 browser specs, 67 simulated runs) and a live test against the pinned opencode that stops a Repair turn and sends the next message. Not yet deployed: the head commit carries `[skip ci]`, so production needs a manual deploy. The apply question that proceeded with two questions unanswered is separate: braid#226.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#225
No description provided.