Every store read is one snapshot, so a stop during the page's first read reaches the run bar (#567) #573

Merged
cmoriarty merged 2 commits from claude/runbar-567 into main 2026-11-20 16:19:27 -05:00
Owner

Two run-bar journeys failed on CI, as runs 569 and 575: a stop and a resume never reached the bar, which counted up from 0s where it should have read 2h. #567 blamed the test's timing. Run 569's trace shows a race in osfd instead:

  • 0.29 s: the page asks for the run's detail, and the busy server takes 210 ms to answer.
  • 0.41 s and 0.50 s: the test's stop and resume commit, while that read is still running.
  • 0.54 s: the stream opens. The page never reads the run again.

Database.read promised one snapshot (docs/design/contract.md §3), but its connection is autocommit, so each query saw the log at its own moment. The run detail read the run's intervals before the stop and its seq after the resume, so it claimed to be as new as the resume. The page trusted that seq and skipped its re-read.

OpenSpec change reads-are-one-snapshot-567, for #567.

What changed

  • Database.read_sync runs its function in one read transaction, so every query sees the same moment of the log. A read inside a read joins it. This covers every API read made of several queries, not only the run detail.
  • The browser tests' server keeps its connections for the one test it serves. This was found while checking: under load, one journey's own POST came 6.5 s after its last request, and uvicorn's 5 s keep-alive had closed the socket (ECONNRESET).
  • docs/testing.md gains a row in the table of flakes: a race that load widens. Read the failed run's trace, and download its artifact before re-running.

Tests

Check On main Here
Store test: a write commits in the middle of a read the read's first query saw seq 1, its second seq 2 both queries agree
Both run-bar journeys, 200 times each, under 12 CPU busy loops with CI's limits 399 of 400 (one ECONNRESET, before the keep-alive fix) 400 of 400
./tools/test.sh passed in 68 s

On the laptop, two cores could not be pinned as #567 asked. The CI image needs a registry login, which would have stored the token.

The rule against sleeps, waits and clocks in tests already exists: the spec requirement test-suite/No test waits on the wall clock, rules 4 and 7 of docs/testing.md, and the checks in tests/test_repo.py and the conftest guard. This flake broke none of them. It was a real race that load widened, which is what the new row in the table says.

🤖 Generated with Claude Code

Two run-bar journeys failed on CI, as runs 569 and 575: a stop and a resume never reached the bar, which counted up from `0s` where it should have read `2h`. #567 blamed the test's timing. Run 569's trace shows a race in osfd instead: - 0.29 s: the page asks for the run's detail, and the busy server takes 210 ms to answer. - 0.41 s and 0.50 s: the test's stop and resume commit, while that read is still running. - 0.54 s: the stream opens. The page never reads the run again. `Database.read` promised one snapshot (`docs/design/contract.md` §3), but its connection is autocommit, so each query saw the log at its own moment. The run detail read the run's intervals before the stop and its seq after the resume, so it claimed to be as new as the resume. The page trusted that seq and skipped its re-read. OpenSpec change `reads-are-one-snapshot-567`, for #567. ## What changed - `Database.read_sync` runs its function in one read transaction, so every query sees the same moment of the log. A read inside a read joins it. This covers every API read made of several queries, not only the run detail. - The browser tests' server keeps its connections for the one test it serves. This was found while checking: under load, one journey's own `POST` came 6.5 s after its last request, and uvicorn's 5 s keep-alive had closed the socket (`ECONNRESET`). - `docs/testing.md` gains a row in the table of flakes: a race that load widens. Read the failed run's trace, and download its artifact before re-running. ## Tests | Check | On main | Here | |---|---|---| | Store test: a write commits in the middle of a read | the read's first query saw seq 1, its second seq 2 | both queries agree | | Both run-bar journeys, 200 times each, under 12 CPU busy loops with CI's limits | 399 of 400 (one `ECONNRESET`, before the keep-alive fix) | 400 of 400 | | `./tools/test.sh` | | passed in 68 s | On the laptop, two cores could not be pinned as #567 asked. The CI image needs a registry login, which would have stored the token. The rule against sleeps, waits and clocks in tests already exists: the spec requirement `test-suite/No test waits on the wall clock`, rules 4 and 7 of `docs/testing.md`, and the checks in `tests/test_repo.py` and the conftest guard. This flake broke none of them. It was a real race that load widened, which is what the new row in the table says. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Every store read runs in one read transaction, as the contract says, so the run detail's seq is
true of what it returns and a stop that lands during the page's first read reaches the bar.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Database.read promised a WAL snapshot (contract §3), but its connection is autocommit, so each query
saw the log as it was when that query ran. On run 569 the run detail read a run's intervals before a
stop committed and the latest seq after the resume, so it claimed to be as new as the resume; the
page, which reads a run again only when the stream says something newer happened, never read it
again, and the bar counted on through the stop. Each read now runs in one read transaction, and a
read inside a read joins it.

Found while checking: under load a journey's own POST came 6.5 s after its last request, and the
test server's uvicorn had closed the idle connection after its default 5 s (ECONNRESET). The test
server lives for one test, so it now keeps its connections.

Tests: a store test where a write commits in the middle of a read (the second query saw it on main);
both run-bar journeys 400 times under CPU load with CI's limits, all passing. docs/testing.md gains a
flake-table row for a race that load widens.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid!573
No description provided.