A fix round stuck on the same failure isn't told to try another approach #360

Closed
opened 2026-11-02 17:11:19 -05:00 by cmoriarty · 1 comment
Owner

What happens
When a round fails the same way as the one before it, the next fix round gets the usual brief and forks the same session, so it starts with the same assumptions. If it fails the same way again, the run stops for a person.

Why
GVS5H's loop tells its worker to "fix it or try a different approach". No Braid brief does. It's rare (3 of 19 October runs had two fix rounds), but each time it works, it saves a stop.

Fix
In that case, add to agent.fix's brief: your last fix didn't change this failure, so say what you assumed, check it, and try a different approach. No new step.

Done when

  • A test shows only a round that failed the same way gets these words.
  • A replay of a stuck fix step (tools/replay.sh) shows the agent trying something else.

Where to look: build_fix_repair in src/osf/engine/prompts.py; src/osf/pipeline/fixloop.py.

From the Memdoor and GVS5H review, 2026-11-02.

**What happens** When a round fails the same way as the one before it, the next fix round gets the usual brief and forks the same session, so it starts with the same assumptions. If it fails the same way again, the run stops for a person. **Why** GVS5H's loop tells its worker to "fix it or try a different approach". No Braid brief does. It's rare (3 of 19 October runs had two fix rounds), but each time it works, it saves a stop. **Fix** In that case, add to `agent.fix`'s brief: your last fix didn't change this failure, so say what you assumed, check it, and try a different approach. No new step. **Done when** - A test shows only a round that failed the same way gets these words. - A replay of a stuck fix step (`tools/replay.sh`) shows the agent trying something else. Where to look: `build_fix_repair` in `src/osf/engine/prompts.py`; `src/osf/pipeline/fixloop.py`. *From the Memdoor and GVS5H review, 2026-11-02.*
Author
Owner

Shipped in PR 362.

A round that failed the same way as the one before it now tells agent.fix: "Your last fix did not change this failure. Before you edit anything, say what you assumed, check it, and try a different approach." A waived round starts the count again.

  • Test: only a round that failed as before gets the words (tests/pipeline/test_fix_stuck_and_visual_gaps.py).
  • Sim: fix-stuck-tries-another.
  • Replay: agent.fix.r2 on a real Free model (reflection/beam-free), 3 of 3 succeeded. Round 1 had only written a note. The replay changed the state itself: "I changed state.txt from "one" to "fixed" so python3 check.py passes".
Shipped in PR 362. A round that failed the same way as the one before it now tells `agent.fix`: "Your last fix did not change this failure. Before you edit anything, say what you assumed, check it, and try a different approach." A waived round starts the count again. - **Test:** only a round that failed as before gets the words (`tests/pipeline/test_fix_stuck_and_visual_gaps.py`). - **Sim:** `fix-stuck-tries-another`. - **Replay:** `agent.fix.r2` on a real Free model (`reflection/beam-free`), 3 of 3 succeeded. Round 1 had only written a note. The replay changed the state itself: "I changed `state.txt` from "one" to "fixed" so `python3 check.py` passes".
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#360
No description provided.