A run installs the repository's dependencies in a scripted step, so agents don't spend their steps hunting for them #109

Closed
opened 2026-09-29 02:36:06 -04:00 by cmoriarty · 1 comment
Owner

What happens

Agents set up the repository's environment themselves, partway through a step. In the 8 runs whose logs are still on production (runs 17 to 29), about 116 of 1,474 bash calls (8%) were dependency hunting: pip install, npm install, which, listing site-packages.

  • ModuleNotFoundError or ImportError came back 36 times, in runs 22, 23, 25 and 26.
  • Run 22 (cmoriarty/scratch, the walking skeleton) couldn't find fastapi and went looking in osfd's own virtualenv at /opt/openspecflow/app/.venv.
  • Run 26 made a .venv in the worktree during openspec.apply[1] to run its tests, which is how #105 happened.
  • soundcheck's node_modules was installed during openspec.apply (#67's step-by-step measurement).

Expected

  • A scripted step installs the repository's dependencies from its manifests, including manifests in subdirectories such as backend/ and frontend/:
    • uv.lock or pyproject.toml → uv sync
    • requirements.txt → a .venv and pip install -r
    • package-lock.json → npm ci
    • and so on for the stacks #107 recognises
  • It runs before openspec.apply. It runs again before the test steps if the run added or changed a manifest, because a new repository has none until apply writes them.
  • The step shows what it ran and how long it took. A failed install fails the step with the installer's output, rather than an agent finding out later.
  • The AGENTS.md block tells agents the environment is ready and where it is, so they use it rather than making their own.

It should use #67's shared npm and uv caches once they land. #105 already keeps these environments out of commits. It's also a precondition for #104's scripted test steps, and for LSP diagnostics if runs ever turn them on (#46).

Adding nodes to the built-in pipelines runs into #102 for runs already in flight, so deploy when idle.

Related: #46, #67, #104, #105, #107.

**What happens** Agents set up the repository's environment themselves, partway through a step. In the 8 runs whose logs are still on production (runs 17 to 29), about 116 of 1,474 bash calls (8%) were dependency hunting: `pip install`, `npm install`, `which`, listing `site-packages`. - `ModuleNotFoundError` or `ImportError` came back 36 times, in runs 22, 23, 25 and 26. - Run 22 (cmoriarty/scratch, the walking skeleton) couldn't find `fastapi` and went looking in osfd's own virtualenv at `/opt/openspecflow/app/.venv`. - Run 26 made a `.venv` in the worktree during `openspec.apply[1]` to run its tests, which is how #105 happened. - soundcheck's `node_modules` was installed during `openspec.apply` (#67's step-by-step measurement). **Expected** - A scripted step installs the repository's dependencies from its manifests, including manifests in subdirectories such as `backend/` and `frontend/`: - `uv.lock` or `pyproject.toml` → `uv sync` - `requirements.txt` → a `.venv` and `pip install -r` - `package-lock.json` → `npm ci` - and so on for the stacks #107 recognises - It runs before `openspec.apply`. It runs again before the test steps if the run added or changed a manifest, because a new repository has none until `apply` writes them. - The step shows what it ran and how long it took. A failed install fails the step with the installer's output, rather than an agent finding out later. - The `AGENTS.md` block tells agents the environment is ready and where it is, so they use it rather than making their own. It should use #67's shared npm and uv caches once they land. #105 already keeps these environments out of commits. It's also a precondition for #104's scripted test steps, and for LSP diagnostics if runs ever turn them on (#46). Adding nodes to the built-in pipelines runs into #102 for runs already in flight, so deploy when idle. Related: #46, #67, #104, #105, #107.
Author
Owner

Shipped in 6a4f98a, as the OpenSpec change install-dependencies. It is now archived (9d27890), and its requirements are the new run-dependencies spec.

Installed before implementation. script.deps runs before openspec.apply (fix.implement in a quick fix) in every implementing built-in pipeline. It finds the manifests git sees, at the root and up to two directories down, and installs each directory with the tool its manifests call for:

  • a lockfile: npm ci;
  • a package.json without one: npm install --no-package-lock;
  • a uv.lock: uv sync --all-extras --all-groups;
  • otherwise, a .venv beside the pyproject.toml or requirements*.txt, with the project installed editable and every extra.

No lockfile the repository lacks is written. An editable install's new *.egg-info/ goes into the run's .git/info/exclude.

Recorded, and a failure says why. .osf/deps.json records each directory with its command, exit code, seconds and a digest of its manifests. A failed install fails the step, and the step's output ends with the installer's own last lines. Go, Rust and other ecosystems are recorded as skipped.

Again before the tests. script.deps.tests installs again only where a manifest is new or changed, so a project written from nothing during implementation is installed before its tests run. The agents' block in AGENTS.md tells them to use these environments rather than make their own.

Verified on production after deploying 3346eae, with run 33 (run_01M3P3GXYAS552B7G7EYPNV8KV, cmoriarty/scratch, git-flow-quick-fix):

  • script.deps installed backend/ with uv venv .venv && uv pip install --python .venv -e . in 11.3 s, and frontend/ with npm ci in 11.1 s.
  • It left git status clean under both directories.
  • script.deps.tests reported that no manifest had changed.
  • None of the run's agents ran pip install, npm install or which, and fix.implement took 17 tool calls.
Shipped in 6a4f98a, as the OpenSpec change `install-dependencies`. It is now archived (9d27890), and its requirements are the new `run-dependencies` spec. **Installed before implementation.** `script.deps` runs before `openspec.apply` (`fix.implement` in a quick fix) in every implementing built-in pipeline. It finds the manifests git sees, at the root and up to two directories down, and installs each directory with the tool its manifests call for: - a lockfile: `npm ci`; - a `package.json` without one: `npm install --no-package-lock`; - a `uv.lock`: `uv sync --all-extras --all-groups`; - otherwise, a `.venv` beside the `pyproject.toml` or `requirements*.txt`, with the project installed editable and every extra. No lockfile the repository lacks is written. An editable install's new `*.egg-info/` goes into the run's `.git/info/exclude`. **Recorded, and a failure says why.** `.osf/deps.json` records each directory with its command, exit code, seconds and a digest of its manifests. A failed install fails the step, and the step's output ends with the installer's own last lines. Go, Rust and other ecosystems are recorded as skipped. **Again before the tests.** `script.deps.tests` installs again only where a manifest is new or changed, so a project written from nothing during implementation is installed before its tests run. The agents' block in `AGENTS.md` tells them to use these environments rather than make their own. **Verified on production** after deploying 3346eae, with run 33 (`run_01M3P3GXYAS552B7G7EYPNV8KV`, cmoriarty/scratch, git-flow-quick-fix): - `script.deps` installed `backend/` with `uv venv .venv && uv pip install --python .venv -e .` in 11.3 s, and `frontend/` with `npm ci` in 11.1 s. - It left `git status` clean under both directories. - `script.deps.tests` reported that no manifest had changed. - None of the run's agents ran `pip install`, `npm install` or `which`, and `fix.implement` took 17 tool calls.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
cmoriarty/braid#109
No description provided.