quick-fix tests the wrong change: a stale .osf/change.json committed to the target repo names someone else's change #98

Closed
opened 2026-09-27 18:28:43 -04:00 by cmoriarty · 1 comment
Owner

What happened

Run run_01M3JCM259FPD9PG96ZRYBN7HK (quick-fix, cmoriarty/soundcheck#29, a docs-only brief) spent its agent.test.unit step (9 min) fixing 57 lint violations in the tests of an unrelated, already-archived change.

The step's prompt said:

## This change
- change id: `admin-artists-durable-not-an-artist-mark-7rq5m0`
- lane: `unit`

and the agent concluded "the .osf brief is stale (a docs run); the step prompt is authoritative", then went to work on the other change's .osf/test/unit.json lint failures.

Why

soundcheck's main tracks the run state of earlier runs: .osf/change.json, .osf/test/unit.json, .osf/test/e2e.json, .osf/review/**, .osf/summary.md, .osf/budget.json, .osf/workspace.json and more (last touched by c65c915 "propose: the change, its specs, its tasks and its estimate").

change_id_or_none (src/osf/engine/prompts.py) deliberately avoids guessing from openspec/changes/, but still trusts .osf/change.json — and in a clone of that repo, the file is the previous run's.

Expected

A run never reads another run's .osf/ state. Either:

  • script.brief (or git.clone) clears per-run files inherited from the base branch before any step reads them, or
  • change_id_or_none ignores .osf/change.json when the pipeline declares uses.openspec: false, and the test/summary prompts read nothing else from .osf/ that the run did not write.

The first fixes every pipeline; the second only quick-fix. Also worth deciding whether per-run state should be committed to the target's main at all.

Acceptance

  • A quick-fix run on a repo whose base branch carries a .osf/change.json from an earlier run gets a test prompt with no change id and no delta specs.
## What happened Run `run_01M3JCM259FPD9PG96ZRYBN7HK` (quick-fix, cmoriarty/soundcheck#29, a docs-only brief) spent its `agent.test.unit` step (9 min) fixing 57 lint violations in the tests of an unrelated, already-archived change. The step's prompt said: ``` ## This change - change id: `admin-artists-durable-not-an-artist-mark-7rq5m0` - lane: `unit` ``` and the agent concluded "the `.osf` brief is stale (a docs run); the step prompt is authoritative", then went to work on the other change's `.osf/test/unit.json` lint failures. ## Why soundcheck's `main` tracks the run state of earlier runs: `.osf/change.json`, `.osf/test/unit.json`, `.osf/test/e2e.json`, `.osf/review/**`, `.osf/summary.md`, `.osf/budget.json`, `.osf/workspace.json` and more (last touched by c65c915 "propose: the change, its specs, its tasks and its estimate"). `change_id_or_none` (`src/osf/engine/prompts.py`) deliberately avoids guessing from `openspec/changes/`, but still trusts `.osf/change.json` — and in a clone of that repo, the file is the previous run's. ## Expected A run never reads another run's `.osf/` state. Either: - `script.brief` (or `git.clone`) clears per-run files inherited from the base branch before any step reads them, or - `change_id_or_none` ignores `.osf/change.json` when the pipeline declares `uses.openspec: false`, and the test/summary prompts read nothing else from `.osf/` that the run did not write. The first fixes every pipeline; the second only quick-fix. Also worth deciding whether per-run state should be committed to the target's `main` at all. ## Acceptance - A quick-fix run on a repo whose base branch carries a `.osf/change.json` from an earlier run gets a test prompt with no change id and no delta specs.
Author
Owner

Shipped in a657dad (change runs-start-without-old-run-state, archived in 55ffb6e).

  • A freshly cloned run worktree drops everything under .osf/ that the base branch carries, except the repository's own .osf/pipeline.yaml and .osf/pipelines/, before any step runs. A worktree reused after a restart is left alone.
  • git.clone says cleared N file(s) an earlier run left in .osf/ on <base>; the deletions ride in the run's first full commit, so a repository's leftover state is replaced rather than accumulating.
  • Fixed for every pipeline, not only quick-fix: test/, review/ and summary.md were inherited too.
  • Tests: fresh clone clears, config kept, restart keeps the run's own state, git.clone reports the count, and the #98 case — a quick-fix unit-test prompt built on a branch tracking another run's change.json names no change. Each fails without the fix. Full lane green (backend 1863, UI unit 444, browser 108).

Note: the built-in git.clone command changed, so any repository with an older committed copy of .osf/pipeline.yaml needs it refreshed (soundcheck has none).

Shipped in a657dad (change `runs-start-without-old-run-state`, archived in 55ffb6e). - A freshly cloned run worktree drops everything under `.osf/` that the base branch carries, except the repository's own `.osf/pipeline.yaml` and `.osf/pipelines/`, before any step runs. A worktree reused after a restart is left alone. - `git.clone` says `cleared N file(s) an earlier run left in .osf/ on <base>`; the deletions ride in the run's first full commit, so a repository's leftover state is replaced rather than accumulating. - Fixed for every pipeline, not only quick-fix: `test/`, `review/` and `summary.md` were inherited too. - Tests: fresh clone clears, config kept, restart keeps the run's own state, `git.clone` reports the count, and the #98 case — a quick-fix unit-test prompt built on a branch tracking another run's `change.json` names no change. Each fails without the fix. Full lane green (backend 1863, UI unit 444, browser 108). Note: the built-in `git.clone` command changed, so any repository with an older committed copy of `.osf/pipeline.yaml` needs it refreshed (soundcheck has none).
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#98
No description provided.