Step time limits cut working agents: replace hard limits with supervision #303
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What happens
Every agent step has a fixed time limit set per kind of step (
timeout_s: code review 1 h, apply 7.5 h). At a tenth of the limit Braid sends a "last call" telling the agent to write its files now, and at the limit it stops the attempt, whatever the agent is doing.Run 139's code review was 47 min in and steadily productive (a browser playtest reporting each check) when the console read "15m left". A limit cuts healthy work as surely as it catches a loop. Models need time as much as anything; mid and small models also fall into hopeless loops, so some watch is still needed.
Ideas to explore
Done when
A proposal compares these on runs 134 to 139's transcripts (time limits hit, loops seen), and picks a design in which a productive step is never stopped for time alone, and a looping one is still caught.
From run 139 (breakout-planned, vLLM). Related: #223, #295 (a slow prefill redone as a stall).
It happened minutes after filing. Run 139's
agent.code-reviewattempt 1:window.__app, reporting each check as it passed.timed_out: "stopped at its 1h time limit;.osf/review/code.jsonis missing or empty".So the review was not looping: it was cut for time, failed, and was restarted under a longer limit, which is a flat limit doubling itself rather than a judgement about the work.
Seen through the engine in simulated runs (PR #316, merged
41f2cad). Before this, only tests of its parts had run the wait loop's progress check. These scenarios all run quick-fix with a 20 s implementer limit:limit-more-time: still writing files at its deadline, the step gotextended_s: 40("still working: 2 file write(s) in the last ten minutes"), finished at 27 s, and the run succeeded. The console's transcript shows⏱ 40s more: still working: 2 file write(s) in the last ten minutes. With the extension switched off, the scenario fails: the step is stopped at 20 s.limit-loop: the same command over and over, nothing written. No more time; stopped at its deadline.limit-backstop: writing every few seconds past three times its limit. More time once, then stopped at the backstop.Not covered: a limit under ten minutes gets no message before its deadline, so these show the check at the deadline, not the one a little before it. This stays open until a step on production shows it. Run 147's steps all ended well inside their limits.
Closing on the simulated runs as proof, the operator's call: "the sim can serve as proof".
limit-more-time,limit-loopandlimit-backstoprun with every simulated set from now on.