Capture a step of a run and replay it on the laptop (#340) #343

Merged
cmoriarty merged 2 commits from claude/replay-340 into main 2026-10-30 13:50:42 -04:00
Owner

For #340; the issue closes once a production capture is replayed on the laptop.

  • Capture: GET /api/steps/{id}/capsule returns one attempt as a tar.gz: the entry checkpoint as a git bundle, the run's fields and frozen settings (no key files or addresses), the models that served each tier, and the transcript.
  • A run that starts at a step: POST /api/runs takes replay: {start_at, only, checkpoint, head, pin}. Admission cuts the graph with loader.from_step, which keeps the dependency steps before the step, and lays the checkpoint over the worktree at head as uncommitted changes. On Free, pin keeps each tier to the models that served the capture.
  • Local replays: tools/replay.sh capture|scripted|free turns a capsule into a sim scenario. scripted plays the captured turns; free runs inside Braid's image, built from the checkout with Docker, against the real router; only the router's container gets the keys from ~/api_keys, never printed. The first Free replay ran on the laptop: the model opened Safari and left servers behind, and its shell could reach any of my files. Also --times, --pin, --through.
  • Docs: docs/testing.md covers replaying a step and closing an issue on evidence from the laptop; CLAUDE.md and sim/README.md point to it.
  • Tests: the cut, the checkpoint restore, the capsule (no key path), the pin, and the transcript as a play.

Checked locally: a sim's Visual check, captured over the API (0.2 s), replayed scripted (8 s, passed) and on Free in the image (5.5 min including the image build: the judge on Qwen 397B drove the browser and looked through Gemini Flash-Lite; passed). Also fixed: Chromium aborted on a socket path over 107 bytes under a long stack folder, so replays run under /tmp/braid-sim-replay-…. ./tools/test.sh passes, and the sims are 76/76. ./tools/test.sh passes.

🤖 Generated with Claude Code

For #340; the issue closes once a production capture is replayed on the laptop. - **Capture:** `GET /api/steps/{id}/capsule` returns one attempt as a tar.gz: the entry checkpoint as a git bundle, the run's fields and frozen settings (no key files or addresses), the models that served each tier, and the transcript. - **A run that starts at a step:** `POST /api/runs` takes `replay: {start_at, only, checkpoint, head, pin}`. Admission cuts the graph with `loader.from_step`, which keeps the dependency steps before the step, and lays the checkpoint over the worktree at `head` as uncommitted changes. On Free, `pin` keeps each tier to the models that served the capture. - **Local replays:** `tools/replay.sh capture|scripted|free` turns a capsule into a sim scenario. `scripted` plays the captured turns; `free` runs inside Braid's image, built from the checkout with Docker, against the real router; only the router's container gets the keys from `~/api_keys`, never printed. The first Free replay ran on the laptop: the model opened Safari and left servers behind, and its shell could reach any of my files. Also `--times`, `--pin`, `--through`. - **Docs:** `docs/testing.md` covers replaying a step and closing an issue on evidence from the laptop; `CLAUDE.md` and `sim/README.md` point to it. - **Tests:** the cut, the checkpoint restore, the capsule (no key path), the pin, and the transcript as a play. Checked locally: a sim's Visual check, captured over the API (0.2 s), replayed scripted (8 s, passed) and on Free in the image (5.5 min including the image build: the judge on Qwen 397B drove the browser and looked through Gemini Flash-Lite; passed). Also fixed: Chromium aborted on a socket path over 107 bytes under a long stack folder, so replays run under `/tmp/braid-sim-replay-…`. `./tools/test.sh` passes, and the sims are 76/76. `./tools/test.sh` passes. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
feat(replay): capture a step of a run and replay it on the laptop (#340)
All checks were successful
test / suite (pull_request) Successful in 35s
50bc43e328
Checking a fix meant waiting on a production run that takes hours. One
attempt of one step can now be captured and replayed locally in minutes.

- GET /api/steps/{id}/capsule: the attempt's entry checkpoint as a git
  bundle, the run's fields and frozen settings (no key files or
  addresses), the models that served each tier, and its transcript.
- POST /api/runs takes `replay: {start_at, only, checkpoint, head, pin}`:
  admission cuts the graph to start at the step (loader.from_step, which
  keeps the dependency steps before it) and lays the checkpoint over the
  worktree at head as uncommitted changes. On Free, `pin` keeps each tier
  to the models that served the capture.
- tools/replay.sh capture|scripted|free: a capsule as a sim scenario;
  scripted plays the captured turns, free runs it through the real router
  with the keys from ~/api_keys (never printed). --times, --pin, --through.
- docs/testing.md and CLAUDE.md: replaying a step, and closing an issue on
  evidence from the laptop instead of a production run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fix(replay): a Free replay runs its agents in Braid's image (#340)
All checks were successful
test / suite (pull_request) Successful in 34s
0a0d27e96a
The first Free replay ran on the laptop: the real model opened Safari,
left http.server processes behind, and its shell could reach any file of
the operator's, the path guard fencing only Braid's state folder.

`tools/replay.sh free` now builds the image from the checkout (cached,
about 50 s), starts the pinned router in a container of its own with the
provider keys passed by name, makes osfd's admin key, and runs each replay
inside the image (`osf.sim.replay inside`), which gets no key. The stack
takes the outside router's address and folder instead of starting one.

A replay's stack lives under /tmp with a short name: Chromium aborted with
"Socket path too long" under /tmp/braid-sim-replay-agent-visual-check-….

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid!343
No description provided.