Run 192 batch: three bugs, and test guidance on clocks and fast browser tests #485

Closed
opened 2026-11-11 11:23:03 -05:00 by cmoriarty · 1 comment
Owner

What happens: run 192 (findings on braid#484) found three Braid bugs and gaps in the test guidance. The owner decided each fix one at a time.

Fix:

Bugs

  1. The text-only test rule (B11) judges each test by what that test itself reads. A name read from a file in one test no longer marks the same name in another.
  2. The code review is offered its reader helpers: add code-review to DELEGATION_WHOLE, with a test that builds the real brief without faking the check.
  3. The saved brief lives in the run's own tmp, so a group working side by side can reread it after a compaction.

Briefs
4. Reword the planner's sentence so running your own browser test is not "running the whole suite", at the same length.
5. In the brief's "The tests you write": no test waits on or times with the real clock. No new lint rule.
6. Research faster browser tests (driving the app by keyboard and its test hooks, headless, reaching a screen without playing through) and add what works to the same section.
7. The requirement limit: the planner's and the repair's briefs say what counts (the text between the heading and the first scenario), including a requirement copied from the main spec. The failure message gives each count, and the help text says the same.

Not done, on the owner's word: checking the hand-over files when they are written; refusing pkill/killall. Side by side waits until agents run in isolated environments (braid#201, braid#203).

Done when: each item has a failing-then-passing test, briefs grow only by the guidance in 5 and 6, and the next run shows the review using reader.

**What happens:** run 192 (findings on braid#484) found three Braid bugs and gaps in the test guidance. The owner decided each fix one at a time. **Fix:** Bugs 1. The text-only test rule (B11) judges each test by what that test itself reads. A name read from a file in one test no longer marks the same name in another. 2. The code review is offered its `reader` helpers: add `code-review` to `DELEGATION_WHOLE`, with a test that builds the real brief without faking the check. 3. The saved brief lives in the run's own `tmp`, so a group working side by side can reread it after a compaction. Briefs 4. Reword the planner's sentence so running your own browser test is not "running the whole suite", at the same length. 5. In the brief's "The tests you write": no test waits on or times with the real clock. No new lint rule. 6. Research faster browser tests (driving the app by keyboard and its test hooks, headless, reaching a screen without playing through) and add what works to the same section. 7. The requirement limit: the planner's and the repair's briefs say what counts (the text between the heading and the first scenario), including a requirement copied from the main spec. The failure message gives each count, and the help text says the same. **Not done, on the owner's word:** checking the hand-over files when they are written; refusing `pkill`/`killall`. Side by side waits until agents run in isolated environments (braid#201, braid#203). **Done when:** each item has a failing-then-passing test, briefs grow only by the guidance in 5 and 6, and the next run shows the review using `reader`.
Author
Owner

Shipped in PR 486 (merge f27905f), archived in PR 487 (merged with [skip ci], no redeploy). Production serves f27905f on a new boot.

  • B11: judges each test by what that test reads. The run 192 false failures, a file's html in one test and rendered markup in another, now pass.
  • Code review helpers: the code review is offered reader. The test builds the real brief with nothing faked.
  • Saved brief: lives in the run's own tmp folder, so a side-by-side group can reread it after compacting.
  • Planner: a task writing a browser test runs that one test. --list only loads it.
  • Test guidance: no waiting on or timing with the real clock (fake timers, page.clock). A browser test reaches its state through the app's test hook, a seed or a shortcut instead of stepping through the whole flow.
  • Requirement length: the planner, the repair, the failure message and the help say what counts toward 500 characters. The repair can run osf.spec_length to list the long ones.

Evidence: each fix has a test that failed before and passes after. ./tools/test.sh and ./tools/sim.sh --smoke passed on the laptop, and CI passed. The next full run should show no B11 false failures, the code review using reader, and no wall-clock tests.

Shipped in PR 486 (merge `f27905f`), archived in PR 487 (merged with `[skip ci]`, no redeploy). Production serves `f27905f` on a new boot. - **B11:** judges each test by what that test reads. The run 192 false failures, a file's `html` in one test and rendered markup in another, now pass. - **Code review helpers:** the code review is offered `reader`. The test builds the real brief with nothing faked. - **Saved brief:** lives in the run's own `tmp` folder, so a side-by-side group can reread it after compacting. - **Planner:** a task writing a browser test runs that one test. `--list` only loads it. - **Test guidance:** no waiting on or timing with the real clock (fake timers, `page.clock`). A browser test reaches its state through the app's test hook, a seed or a shortcut instead of stepping through the whole flow. - **Requirement length:** the planner, the repair, the failure message and the help say what counts toward 500 characters. The repair can run `osf.spec_length` to list the long ones. Evidence: each fix has a test that failed before and passes after. `./tools/test.sh` and `./tools/sim.sh --smoke` passed on the laptop, and CI passed. The next full run should show no B11 false failures, the code review using `reader`, and no wall-clock tests.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#485
No description provided.