Agents run the slow test suite: they are never told the project's fast test command #65
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Problem
On run
run_01M3FS9FFXEHTY5WTHX36V4Z5Y(issue #52,fix.implementattempt 3), the step'stestsubagent ranuv run pytest tests/ -qand was still at about 52% after 15 minutes: one process, every slow test, andtests/integrationtoo. This repository has had a fast lane since #39../tools/test.shrunspytest -n auto -m "not slow" tests/unitacross every core, plus the UI checks, in about a minute. The agent never used it.Two causes, one in this repository and one in Braid:
CLAUDE.md. opencode readsAGENTS.md, and it readsCLAUDE.mdonly when there is noAGENTS.md. This repository'sAGENTS.mdholds nothing but Braid's generated block, which says nothing about tests. So Braid's agents never learn the lane exists.openspec.test.*prompt offersuv run pytest -q tests/unitas its example: no parallelism, no slow-test filter. Nothing in the generatedAGENTS.mdblock, or in the delegation guidance for thetestsubagent, tells an agent to look for and use the project's own documented test command. So in any repository an agent picks whateverpytest/npm testcall it thinks of first.The same guessed command also ends up in
.osf/test/unit.command, which the orchestrator then runs itself to judge the lane, so the cost is paid twice.Fix
AGENTS.md, outside Braid's block, giving the fast commands and saying not to runpytest tests/bare.tools/test.shitself is not the one to hand agents: its browser step shares a fixed fixture port (8719), and two concurrent runs would test each other's builds.AGENTS.mdblock says to use the project's own documented test command (AGENTS.md,CLAUDE.md, README, CONTRIBUTING, a test script, a Makefile target), to run in parallel where the project supports it, and to leave slow and integration suites to the steps that own them;.osf/test/<lane>.command, and drop the misleading example;testsubagent guidance in the delegation policy says to give it the project's fast command.Shipped in #66 (
e224078, archived as2026-09-26-agents-use-project-test-command).AGENTS.mdblock now tells every agent to use the project's own documented test command, to hand it to atestsubagent, to run in parallel where it can, and to leave slow, integration and end-to-end suites to their own steps.script.agents-mdkeeps its block current: it replaces the text between its markers when it differs, and leaves the rest ofAGENTS.mdbyte for byte. Repositories that merged an earlier run's block will get the new one on their next run..osf/test/<lane>.command, and the misleadinguv run pytest -q tests/unitexample is gone.AGENTS.mdhas "Running the tests". The unit line (uv run pytest -q -n auto -m "not slow" tests/unit --deselect tests/unit/test_metrics.py && cd ui && npx tsc -b && npx vitest run) runs 1,772 backend and 410 UI tests in 36 s, against 30+ minutes forpytest tests/.A run already in flight keeps the
AGENTS.mdit was scaffolded with.