Remove lint rule B7, and keep advisory rules out of the briefs #294

Closed
opened 2026-10-28 19:51:48 -04:00 by cmoriarty · 1 comment
Owner

What happens

Lint rule B7 ("the test file imports the module under test") misfired three times in #279, each time costly, though it only advises and never blocks:

  • Run 133 renamed a test twice to match the name B7 guessed.
  • Run 134 renamed its source file app.js → mortgage.js for the same reason.
  • Run 136's spec-test step: B7 missed const app = await import('../app.js'), so the agent moved code into a helper, dom-harness.mjs; the step's test-only check refused it, and the run failed after 905 s.

It also flags every browser spec and any black-box test of an app over HTTP or the command line. What it means to catch (a test asserting on its own data) is caught elsewhere: B1 (mocks), the lanes running the tests against the real code, the scenario check, and the judge.

Agents act on any rule they're shown, advice included.

Fix

  • Remove B7 from the lint, its brief text and the docs.
  • Show agents only the rules that block (B1, B2, B6). Keep the advisory ones (B3, B4, B5) in the round's record, for people.
  • Let the spec-test step's test-only check accept a helper that only tests import.

Done when

ban-check has no B7, no brief lists an advisory rule, and a spec-test step that adds a test-only helper succeeds.

From #279, runs 133, 134 and 136. Replaces #278.

**What happens** Lint rule B7 ("the test file imports the module under test") misfired three times in #279, each time costly, though it only advises and never blocks: - Run 133 renamed a test twice to match the name B7 guessed. - Run 134 renamed its source file `app.js` → `mortgage.js` for the same reason. - Run 136's spec-test step: B7 missed `const app = await import('../app.js')`, so the agent moved code into a helper, `dom-harness.mjs`; the step's test-only check refused it, and the run failed after 905 s. It also flags every browser spec and any black-box test of an app over HTTP or the command line. What it means to catch (a test asserting on its own data) is caught elsewhere: B1 (mocks), the lanes running the tests against the real code, the scenario check, and the judge. Agents act on any rule they're shown, advice included. **Fix** - Remove B7 from the lint, its brief text and the docs. - Show agents only the rules that block (B1, B2, B6). Keep the advisory ones (B3, B4, B5) in the round's record, for people. - Let the spec-test step's test-only check accept a helper that only tests import. **Done when** `ban-check` has no B7, no brief lists an advisory rule, and a spec-test step that adds a test-only helper succeeds. _From #279, runs 133, 134 and 136. Replaces #278._
cmoriarty changed title from A false B7 finding makes the spec-test step add a helper, which fails the run to Remove lint rule B7, and keep advisory rules out of the briefs 2026-10-28 19:54:23 -04:00
Author
Owner

Shipped in PR #297 (1d98b25). B7 is gone from the lint, its brief text and the guide. ban-check prints only the rules that block (B1, B2, B6), --all for every rule, and the briefs list only those. The test-only refusal now ends "a test's helper goes under a test directory, such as test/", and the scenario-test brief says the same. Tests: a dynamic import(), a helper import, a Playwright-only spec and a file with no imports all lint clean.

Verified on run 137 (llama.cpp): apply ran ban-check once and it printed []. agent.write-spec-tests, the step run 136 failed on B7, passed first time: its ban-check printed one B6 finding (a blocking rule), the agent tagged the test, and the next check printed []. No agent mentioned a recorded rule.

Shipped in PR #297 (1d98b25). B7 is gone from the lint, its brief text and the guide. `ban-check` prints only the rules that block (B1, B2, B6), `--all` for every rule, and the briefs list only those. The test-only refusal now ends "a test's helper goes under a test directory, such as test/", and the scenario-test brief says the same. Tests: a dynamic `import()`, a helper import, a Playwright-only spec and a file with no imports all lint clean. Verified on run 137 (llama.cpp): apply ran `ban-check` once and it printed `[]`. `agent.write-spec-tests`, the step run 136 failed on B7, passed first time: its `ban-check` printed one B6 finding (a blocking rule), the agent tagged the test, and the next check printed `[]`. No agent mentioned a recorded rule.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#294
No description provided.