Agents can read Braid's own code, and have no documentation of Braid #532

Closed
opened 2026-11-13 15:07:32 -05:00 by cmoriarty · 1 comment
Owner

What happens

A step's sandbox holds Braid's whole program folder read-only, because the brief tells the agent to run Braid's test lint inside it. Run 194's first group read 170 lines of the lint through sed before writing a test (braid#501, finding 1). The guard has refused a shell command that names the package since braid#506, but it reads a command's text, so a computed path still reaches it. Several findings of runs 193 to 195 were an agent not knowing how Braid works here: where screenshots go, where the browsers are, how to stop a command left in the background.

Fix (owner, 2026-11-12: "the fact that it has access to something we didn't plan for is a problem that should be fixed"; "agents should have access to braid documentation")

  • Take Braid's package out of the sandbox: the lint becomes a tool Braid runs outside, as app is. First find what else in the sandbox needs that folder (the interpreter, the shell wrapper).
  • A small read-only folder of pages written for agents, with one line in the brief pointing at it: the lint's rules, lanes and checks, the helpers, screenshots and temporary files, stopping a background command by the id its shell gave.

Done when

A step's shell cannot read Braid's package by any path, the lint still runs, and the pages are in every sandbox.

**What happens** A step's sandbox holds Braid's whole program folder read-only, because the brief tells the agent to run Braid's test lint inside it. Run 194's first group read 170 lines of the lint through `sed` before writing a test (braid#501, finding 1). The guard has refused a shell command that names the package since braid#506, but it reads a command's text, so a computed path still reaches it. Several findings of runs 193 to 195 were an agent not knowing how Braid works here: where screenshots go, where the browsers are, how to stop a command left in the background. **Fix** (owner, 2026-11-12: "the fact that it has access to something we didn't plan for is a problem that should be fixed"; "agents should have access to braid documentation") - Take Braid's package out of the sandbox: the lint becomes a tool Braid runs outside, as `app` is. First find what else in the sandbox needs that folder (the interpreter, the shell wrapper). - A small read-only folder of pages written for agents, with one line in the brief pointing at it: the lint's rules, lanes and checks, the helpers, screenshots and temporary files, stopping a background command by the id its shell gave. **Done when** A step's shell cannot read Braid's package by any path, the lint still runs, and the pages are in every sandbox.
Author
Owner

Shipped in pull request 566 (change sandbox-edges-493-532).

  • Braid's program is in no sandbox. Every step sandbox hides Braid's folder (/opt/openspecflow/app in the image). The spawner and the shell wrapper run on the image's own interpreter from copies outside it, and no shell gets OSF_PYTHON.
  • The braid tool runs Braid's checks in the run's server, beside the sandbox: lint, check-report, plan-check, plan-check-replan, spec-length. The briefs name it. A shell command that runs $OSF_PYTHON is refused, naming the call that does the same.
  • Pages for agents: every run holds a read-only braid-docs/ (an index, the lint's rules and the checks generated from their own help, lanes, helpers, files and screenshots, background commands), named in one line at the end of every agent brief.

Evidence: in the image built from the branch, with the stack's sandbox options, a step sandbox lists Braid's folder empty, import osf fails and a search for its code finds nothing, while the shell and Playwright's browsers work. braid-selfcheck passes with its simulated runs. Tests cover the tool, its refusals, the lint's help and the pages.

On production since 19a75bd: the deploy's self-check in the image passed, with its 6 simulated runs fenced, and osfd's checks of the fence and the step sandbox all passed. Archived in pull request 568.

Shipped in pull request 566 (change `sandbox-edges-493-532`). - **Braid's program is in no sandbox.** Every step sandbox hides Braid's folder (`/opt/openspecflow/app` in the image). The spawner and the shell wrapper run on the image's own interpreter from copies outside it, and no shell gets `OSF_PYTHON`. - **The `braid` tool** runs Braid's checks in the run's server, beside the sandbox: `lint`, `check-report`, `plan-check`, `plan-check-replan`, `spec-length`. The briefs name it. A shell command that runs `$OSF_PYTHON` is refused, naming the call that does the same. - **Pages for agents:** every run holds a read-only `braid-docs/` (an index, the lint's rules and the checks generated from their own help, lanes, helpers, files and screenshots, background commands), named in one line at the end of every agent brief. **Evidence:** in the image built from the branch, with the stack's sandbox options, a step sandbox lists Braid's folder empty, `import osf` fails and a search for its code finds nothing, while the shell and Playwright's browsers work. `braid-selfcheck` passes with its simulated runs. Tests cover the tool, its refusals, the lint's help and the pages. On production since 19a75bd: the deploy's self-check in the image passed, with its 6 simulated runs fenced, and osfd's checks of the fence and the step sandbox all passed. Archived in pull request 568.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#532
No description provided.