A hand-over file in the wrong shape counts for nothing instead of failing the scenarios step #440

Closed
opened 2026-11-08 15:59:40 -05:00 by cmoriarty · 1 comment
Owner

What happens. Run 183's apply wrote .osf/test/visual.json as {"summary": …, "visual": [{"spec", "requirement", "scenario", "reason"}]} and .osf/test/untested.json as {"summary": …, "untested": [{"spec", "requirement", "scenarios": […], "reason"}]}. Braid reads {"scenarios": [{"id": "<capability>/<scenario>", "why": …}]} and nothing else, so both files counted for nothing: 49 scenarios uncovered, none left to the Visual check, none untested, no word to anyone. The code review's brief got no untested list; the Visual check is told to judge every uncovered scenario a person could see (braid#439, finding 6).

Fix. script.scenarios fails when either file exists and is not the documented shape (no scenarios list, or an entry without id and why), naming the file, what it found and the shape wanted, so the apply (or the fix round) rewrites it; the record carries the error. The brief's line on the shape stays as it is.

Done when. A unit test on run 183's two files fails the step with the message; a well-formed file passes as today; a sim scenario with a mis-shaped visual.json shows the step red with the message.

From run 183 (braid#439).

**What happens.** Run 183's apply wrote `.osf/test/visual.json` as `{"summary": …, "visual": [{"spec", "requirement", "scenario", "reason"}]}` and `.osf/test/untested.json` as `{"summary": …, "untested": [{"spec", "requirement", "scenarios": […], "reason"}]}`. Braid reads `{"scenarios": [{"id": "<capability>/<scenario>", "why": …}]}` and nothing else, so both files counted for nothing: 49 scenarios uncovered, none left to the Visual check, none untested, no word to anyone. The code review's brief got no untested list; the Visual check is told to judge every uncovered scenario a person could see (braid#439, finding 6). **Fix.** `script.scenarios` fails when either file exists and is not the documented shape (no `scenarios` list, or an entry without `id` and `why`), naming the file, what it found and the shape wanted, so the apply (or the fix round) rewrites it; the record carries the error. The brief's line on the shape stays as it is. **Done when.** A unit test on run 183's two files fails the step with the message; a well-formed file passes as today; a sim scenario with a mis-shaped `visual.json` shows the step red with the message. From run 183 (braid#439).
Author
Owner

Fixed in PR 448, though not as the issue first put it.

What changed. A new completion check, scenario-lists, on the apply steps and the fix rounds. When either file exists in another shape, the step that wrote it fails its check, with the file, what it holds and the shape Braid reads. The step is tried again in its own session. The round's record carries handover_errors.

Why not script.scenarios failing. A scripted step that fails stops the run for a person, hours in, over a file an agent rewrites in a minute. The apply's session is still there when its own check runs.

Evidence from the laptop.

  • A unit test on run 183's two shapes: `.osf/test/visual.json` has no `scenarios` list (its keys: `summary`, `visual`) and the same for untested.json, followed by the shape to write.
  • An entry with no id, with no why, and one that is not an object are each named. A sound file passes, and so does a step that wrote neither.
  • New sim apply-misshaped-handover: the apply's first attempt writes run 183's shape, its check fails with the message, the retry rewrites the file, the Visual check is left the scenario, and the run merges.

Not done: a replay of run 183's apply. Scripted, it holds only the step's first session, which ends before the files are written. The next Scratch run is the soak test.

Fixed in PR 448, though not as the issue first put it. **What changed.** A new completion check, `scenario-lists`, on the apply steps and the fix rounds. When either file exists in another shape, the step that wrote it fails its check, with the file, what it holds and the shape Braid reads. The step is tried again in its own session. The round's record carries `handover_errors`. **Why not `script.scenarios` failing.** A scripted step that fails stops the run for a person, hours in, over a file an agent rewrites in a minute. The apply's session is still there when its own check runs. **Evidence from the laptop.** - A unit test on run 183's two shapes: `` `.osf/test/visual.json` has no `scenarios` list (its keys: `summary`, `visual`) `` and the same for `untested.json`, followed by the shape to write. - An entry with no `id`, with no `why`, and one that is not an object are each named. A sound file passes, and so does a step that wrote neither. - New sim `apply-misshaped-handover`: the apply's first attempt writes run 183's shape, its check fails with the message, the retry rewrites the file, the Visual check is left the scenario, and the run merges. Not done: a replay of run 183's apply. Scripted, it holds only the step's first session, which ends before the files are written. The next Scratch run is the soak test.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#440
No description provided.