Step replays fail: production capture, and local Free replays #581

Closed
opened 2026-11-20 19:46:02 -05:00 by cmoriarty · 1 comment
Owner

What happens

  • Capturing a step on production fails. GET /api/steps/<id>/capsule answers 409, "could not bundle the checkpoint: Unable to create '/tmp/tmpXXXX/repo.bundle.lock'", for the summary steps of run 204 and run 203. Since braid#493, osfd runs git for a run's folder in a sandbox with a /tmp of its own, so the bundle path osfd made is not there.
  • A local Free replay never starts its agent. Braid's image sets OSF_SANDBOX=step, Docker Desktop's kernel has no Landlock, and the run's server is refused. The step is then held as engine_down, and the replay waits until its wall time without saying why. A stopped replay also leaves its router's container running.

Fix

  • Write the bundle to git's standard output.
  • Start the replay's container as production starts Braid's: its seccomp profile and /dev/net/tun, with OSF_SANDBOX=step-unfenced.
  • End a replay at once when its run's server cannot start, saying why, and remove the router's container when a replay is stopped.

Done when a production step can be captured, and replayed on Free on the laptop.

**What happens** - Capturing a step on production fails. `GET /api/steps/<id>/capsule` answers 409, "could not bundle the checkpoint: Unable to create '/tmp/tmpXXXX/repo.bundle.lock'", for the summary steps of run 204 and run 203. Since braid#493, osfd runs git for a run's folder in a sandbox with a `/tmp` of its own, so the bundle path osfd made is not there. - A local Free replay never starts its agent. Braid's image sets `OSF_SANDBOX=step`, Docker Desktop's kernel has no Landlock, and the run's server is refused. The step is then held as `engine_down`, and the replay waits until its wall time without saying why. A stopped replay also leaves its router's container running. **Fix** - Write the bundle to git's standard output. - Start the replay's container as production starts Braid's: its seccomp profile and `/dev/net/tun`, with `OSF_SANDBOX=step-unfenced`. - End a replay at once when its run's server cannot start, saying why, and remove the router's container when a replay is stopped. **Done when** a production step can be captured, and replayed on Free on the laptop.
Author
Owner

Shipped in #583 (merged as c9c5e63, live on production), and archived in #584.

  • The capsule's bundle comes from git's standard output. After the deploy, run 204's summary step was captured from production; it answered 409 before.
  • A Free replay's container runs as production's does, with OSF_SANDBOX=step-unfenced. Replays of run 204's summary step and of a simulated run's ran to their end on Free models.
  • A run whose server cannot start ends a replay or a simulated run within seconds, with osfd's reason. A stopped replay removes its containers.
  • Hetzner is unused: gomodel 0.1.103's built-in Hetzner provider reads /api/v1, which Hetzner no longer serves (/v1 does), so its model was never found. The catalog has the new address, ready for a router that reads it.
Shipped in #583 (merged as `c9c5e63`, live on production), and archived in #584. - The capsule's bundle comes from git's standard output. After the deploy, run 204's summary step was captured from production; it answered 409 before. - A Free replay's container runs as production's does, with `OSF_SANDBOX=step-unfenced`. Replays of run 204's summary step and of a simulated run's ran to their end on Free models. - A run whose server cannot start ends a replay or a simulated run within seconds, with osfd's reason. A stopped replay removes its containers. - Hetzner is unused: gomodel 0.1.103's built-in Hetzner provider reads `/api/v1`, which Hetzner no longer serves (`/v1` does), so its model was never found. The catalog has the new address, ready for a router that reads it.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#581
No description provided.