An apply step forgets .osf/test/unit.command: the rule is one conditional clause, 60,000 characters before the end of its brief #144

New issue

Closed

opened 2026-10-05 11:32:58 -04:00 by cmoriarty · 1 comment

cmoriarty commented

2026-10-05 11:32:58 -04:00

Owner

In run 47, openspec.apply[1]'s first attempt did group 1's work correctly: the edits, the test assertions, the full suite (100 passed) and the tasks ticked. But it never wrote .osf/test/unit.command. The step's completion check (lane-declared:unit, added by #139 so the fix loop's unit lane has a command) failed with "the unit lane has no command". The retry wrote the file in 56 seconds. The recovery worked, but the first attempt should not have missed it.

Why the model missed it

The brief is 65,345 characters long. The rule appears once, at character 5,188, as a clause: "If that file is not there yet, write it". Everything after it is about the tasks: the group's own tasks with their own checks, the review findings, the whole change, and 16,000 characters of room.py.
The brief ends with the generic last line many steps share: "When the file is written, reply with one short paragraph…". An apply step writes no single file, and the line says nothing about the command.
Runs 44 and 46 wrote the file on their first attempt. Run 47 didn't.

The same section also has stray runs of spaces ("records their exit codes", "not there yet"). Its wrapped string literals keep their indentation when they are joined.

Asked for

When the file is missing, the brief says so as a fact: .osf/test/unit.command does not exist yet, and this group writes it. When the file exists, the brief names the command it holds.
An apply step's brief ends by saying what done is: the group's tasks ticked and the unit command in place. Then it asks for the reply.
A quick fix's implementer gets the same, since it has the same check.
The briefs have no stray runs of spaces.

In run 47, `openspec.apply[1]`'s first attempt did group 1's work correctly: the edits, the test assertions, the full suite (100 passed) and the tasks ticked. But it never wrote `.osf/test/unit.command`. The step's completion check (`lane-declared:unit`, added by #139 so the fix loop's unit lane has a command) failed with "the unit lane has no command". The retry wrote the file in 56 seconds. The recovery worked, but the first attempt should not have missed it. **Why the model missed it** - The brief is 65,345 characters long. The rule appears once, at character 5,188, as a clause: "If that file is not there yet, write it". Everything after it is about the tasks: the group's own tasks with their own checks, the review findings, the whole change, and 16,000 characters of `room.py`. - The brief ends with the generic last line many steps share: "When the file is written, reply with one short paragraph…". An apply step writes no single file, and the line says nothing about the command. - Runs 44 and 46 wrote the file on their first attempt. Run 47 didn't. The same section also has stray runs of spaces ("records their exit codes", "not there yet"). Its wrapped string literals keep their indentation when they are joined. **Asked for** - When the file is missing, the brief says so as a fact: `.osf/test/unit.command` does not exist yet, and this group writes it. When the file exists, the brief names the command it holds. - An apply step's brief ends by saying what done is: the group's tasks ticked and the unit command in place. Then it asks for the reply. - A quick fix's implementer gets the same, since it has the same check. - The briefs have no stray runs of spaces.

cmoriarty referenced this issue from a commit

2026-10-05 14:57:47 -04:00

Propose unit-command-in-the-brief (#144) [skip ci]

cmoriarty referenced this issue from a commit

2026-10-05 14:57:47 -04:00

fix(prompts): an apply step is told the unit command is missing, and its brief ends with what done is (#144)

cmoriarty referenced this issue from a commit

2026-10-05 17:23:04 -04:00

Archive unit-command-in-the-brief (#144) [skip ci]

cmoriarty commented

2026-10-05 17:23:25 -04:00

Author

Owner

Shipped, deployed and archived.

What changed (b6df70e; archived in da6efe7)

An apply step's brief reads .osf/test/unit.command as it is built. When the file is missing, the brief says so as a fact, and that this step writes it, even when the step's own tasks add no test. When the file is there, the brief names the command it holds and does not ask for it.
The apply and quick-fix briefs end by saying what done is: the group's tasks ticked (or .osf/fix.md with its Acceptance section) and, when it was missing, .osf/test/unit.command written. Only then do they ask for the reply. The generic "When the file is written" ending is gone from both.
Wrapped text in prompts.py no longer joins into runs of spaces: 234 continuation lines lost their leading space, and a source test fails on any join that makes a run.
Specs: step-briefs and quick-fix-pipeline are updated, and step-briefs has a new requirement, "A brief's own text has no stray runs of spaces".

Verified

test_prompts.py: a missing, present, empty and scripted command, for a change and for a quick fix; the endings; and the briefs rendered before and after the change match once runs of spaces are collapsed.
Production, run 48: openspec.apply[1]'s brief said the file did not exist yet and ended by naming it. The step's first attempt wrote python -m pytest backend/tests && npm --prefix frontend test, read it back, and succeeded in 15 minutes. In run 47, the same step's first attempt had failed its check for want of the file.

Shipped, deployed and archived. **What changed** (b6df70e; archived in da6efe7) - An apply step's brief reads `.osf/test/unit.command` as it is built. When the file is missing, the brief says so as a fact, and that this step writes it, even when the step's own tasks add no test. When the file is there, the brief names the command it holds and does not ask for it. - The apply and quick-fix briefs end by saying what done is: the group's tasks ticked (or `.osf/fix.md` with its Acceptance section) and, when it was missing, `.osf/test/unit.command` written. Only then do they ask for the reply. The generic "When the file is written" ending is gone from both. - Wrapped text in `prompts.py` no longer joins into runs of spaces: 234 continuation lines lost their leading space, and a source test fails on any join that makes a run. - Specs: `step-briefs` and `quick-fix-pipeline` are updated, and `step-briefs` has a new requirement, "A brief's own text has no stray runs of spaces". **Verified** - `test_prompts.py`: a missing, present, empty and scripted command, for a change and for a quick fix; the endings; and the briefs rendered before and after the change match once runs of spaces are collapsed. - Production, run 48: `openspec.apply[1]`'s brief said the file did not exist yet and ended by naming it. The step's first attempt wrote `python -m pytest backend/tests && npm --prefix frontend test`, read it back, and succeeded in 15 minutes. In run 47, the same step's first attempt had failed its check for want of the file.

cmoriarty closed this issue

2026-10-05 17:23:25 -04:00

cmoriarty referenced this issue

2026-10-05 19:31:30 -04:00

A pipeline can only be tested by a real run: simulate whole runs in minutes, with a scripted model, locally and in the deploy's image #149

cmoriarty referenced this issue from a commit

2026-10-05 22:08:23 -04:00

feat(sim): a shipped pipeline runs end to end in seconds, with a scripted model and a fake forge (#149)

cmoriarty referenced this issue

2026-10-05 22:11:51 -04:00

A run's tests and fixes are spread over five agent sessions, and none of them judges the result: a hello world took 74 minutes and did not work #139

cmoriarty referenced this issue