A docs pipeline: prose and comments to a pull request, with no tests and no OpenSpec change #99

Closed
opened 2026-09-27 18:29:44 -04:00 by cmoriarty · 1 comment
Owner

Why

cmoriarty/soundcheck#29 ("Document the admin console") is docs only, and ran on quick-fix (run_01M3JCM259FPD9PG96ZRYBN7HK). The docs themselves came out well — a README section, a 151-line docs/admin-console.md, a cross-reference in docs/deploy.md — but the pipeline is built for code:

  • fix.implement took 23 min. Its prompt says "a test for it must fail without the fix", so it also wrote src/docs.test.ts: 9 assertions that the README contains certain strings. It also touched package-lock.json.
  • agent.test.unit then ran for 9 min (and went wrong — see #98). On soundcheck the full npm test is ~6 min and ~8 GB.
  • fix.summarize / git.commit.summary are a second agent step to write what the first already knows.

Proposal: a docs built-in

git.clone → script.brief → script.agents-md → docs.write → docs.check
  → git.commit.feature → git.pr.open → human.merge-pr → git.pr.merge

9 nodes (quick-fix has 13), one agent step (quick-fix has up to four).

docs.write (agent, timeout 1800 s, not 5400)

  • Never adds tests. Every factual claim (path, URL, env var, route, command, file:line) must be read from the source, not taken from the brief.
  • Writes .osf/summary.md itself, in PR form: what changed, what to check, and that no code or spec changed. That drops fix.summarize and git.commit.summary.
  • Completion: paths-exist:.osf/summary.md, worktree-changed, and a new docs-only.

docs-only predicate — every changed path outside .osf/ is either

  • prose: *.md, *.mdx, *.rst, *.txt, docs/**, LICENSE, CHANGELOG; or
  • a source file whose added and removed lines are all blank or comments (#, //, /* */, *, <!-- -->).

Otherwise the step fails with "code changed — use quick-fix". This would have caught src/docs.test.ts and the lockfile.

docs.check (scripted, no model) — relative Markdown links resolve, and cited file:line references point at files and lines that exist. No test suite: once docs-only holds, there is nothing for tests to exercise.

Left out on purpose: no review agent (the merge gate is the review), no e2e or browser steps (ui_changed is false by construction).

Entry points: offered in the New Run picker, started by the osf:docs label, plus git-flow-docs (the same shape on _FEATURE branches) — as quick-fix does.

Where it goes

  • src/osf/pipeline/nodes.py: a _DOCS selection, docs.write and docs.check in _EXTRA_NODES, docs and git-flow-docs in _BUILTIN_SHAPES.
  • src/osf/engine/prompts.py: build_docs_write.
  • src/osf/engine/predicates.py: docs-only.
  • A link / file:line checker for docs.check.

Open question

Python docstrings are not comments by the line rule. Proposed: treat them as code, so docstring-only changes go to quick-fix. Covering them needs an AST comparison (the module's AST with docstrings stripped is unchanged).

## Why cmoriarty/soundcheck#29 ("Document the admin console") is docs only, and ran on `quick-fix` (`run_01M3JCM259FPD9PG96ZRYBN7HK`). The docs themselves came out well — a README section, a 151-line `docs/admin-console.md`, a cross-reference in `docs/deploy.md` — but the pipeline is built for code: - `fix.implement` took 23 min. Its prompt says "a test for it must fail without the fix", so it also wrote `src/docs.test.ts`: 9 assertions that the README contains certain strings. It also touched `package-lock.json`. - `agent.test.unit` then ran for 9 min (and went wrong — see #98). On soundcheck the full `npm test` is ~6 min and ~8 GB. - `fix.summarize` / `git.commit.summary` are a second agent step to write what the first already knows. ## Proposal: a `docs` built-in ``` git.clone → script.brief → script.agents-md → docs.write → docs.check → git.commit.feature → git.pr.open → human.merge-pr → git.pr.merge ``` 9 nodes (quick-fix has 13), one agent step (quick-fix has up to four). **`docs.write`** (agent, timeout 1800 s, not 5400) - Never adds tests. Every factual claim (path, URL, env var, route, command, `file:line`) must be read from the source, not taken from the brief. - Writes `.osf/summary.md` itself, in PR form: what changed, what to check, and that no code or spec changed. That drops `fix.summarize` and `git.commit.summary`. - Completion: `paths-exist:.osf/summary.md`, `worktree-changed`, and a new `docs-only`. **`docs-only` predicate** — every changed path outside `.osf/` is either - prose: `*.md`, `*.mdx`, `*.rst`, `*.txt`, `docs/**`, `LICENSE`, `CHANGELOG`; or - a source file whose added and removed lines are all blank or comments (`#`, `//`, `/* */`, `*`, `<!-- -->`). Otherwise the step fails with "code changed — use quick-fix". This would have caught `src/docs.test.ts` and the lockfile. **`docs.check`** (scripted, no model) — relative Markdown links resolve, and cited `file:line` references point at files and lines that exist. No test suite: once `docs-only` holds, there is nothing for tests to exercise. **Left out on purpose:** no review agent (the merge gate is the review), no e2e or browser steps (`ui_changed` is false by construction). **Entry points:** offered in the New Run picker, started by the `osf:docs` label, plus `git-flow-docs` (the same shape on `_FEATURE` branches) — as quick-fix does. ## Where it goes - `src/osf/pipeline/nodes.py`: a `_DOCS` selection, `docs.write` and `docs.check` in `_EXTRA_NODES`, `docs` and `git-flow-docs` in `_BUILTIN_SHAPES`. - `src/osf/engine/prompts.py`: `build_docs_write`. - `src/osf/engine/predicates.py`: `docs-only`. - A link / `file:line` checker for `docs.check`. ## Open question Python docstrings are not comments by the line rule. Proposed: treat them as code, so docstring-only changes go to quick-fix. Covering them needs an AST comparison (the module's AST with docstrings stripped is unchanged).
Author
Owner

Shipped in 2acc7ac (change docs-pipeline, archived in c623806).

  • New built-ins docs and git-flow-docs: git.clone → script.brief → script.agents-md → docs.write → docs.check → git.commit.feature → git.pr.open → human.merge-pr → git.pr.merge. One agent step; no test lanes, no browser check, no OpenSpec change, no archive.
  • docs.write forbids tests, requires every path/URL/env var/route/file:line to be read from the repository, and writes .osf/summary.md as the PR description itself (no summary step). Timeout 1800 s.
  • docs-only completion check: every changed path outside .osf/ is prose (.md .mdx .rst .txt .adoc, docs/**, README/LICENSE/CHANGELOG/…) or a source file whose changed lines are all blank or whole-line comments. A test, lockfile, docstring, trailing comment or deleted source file fails it, naming the file and pointing at quick-fix.
  • docs.check (scripted, python -m osf.pipeline.docs check): relative Markdown links and backticked path:N[-M] on lines the run added must resolve; old broken links are ignored, URLs aren't fetched. Report in .osf/docs-check.json.
  • Offered in the New Run wizard wherever quick-fix is; osf:docs starts one. The label must be created by hand on each repository, like osf:quick-fix.
  • Specs: new docs-pipeline; named-pipelines and new-run updated. Guide updated. Full lane green (backend 1883, UI unit 445, browser 108).

To retry cmoriarty/soundcheck#29 on it: redeploy osfd, create osf:docs on soundcheck, and label the issue. #98's fix ships in the same deploy, so the run won't inherit the old .osf/change.json.

Shipped in 2acc7ac (change `docs-pipeline`, archived in c623806). - New built-ins `docs` and `git-flow-docs`: `git.clone → script.brief → script.agents-md → docs.write → docs.check → git.commit.feature → git.pr.open → human.merge-pr → git.pr.merge`. One agent step; no test lanes, no browser check, no OpenSpec change, no archive. - `docs.write` forbids tests, requires every path/URL/env var/route/`file:line` to be read from the repository, and writes `.osf/summary.md` as the PR description itself (no summary step). Timeout 1800 s. - `docs-only` completion check: every changed path outside `.osf/` is prose (`.md .mdx .rst .txt .adoc`, `docs/**`, README/LICENSE/CHANGELOG/…) or a source file whose changed lines are all blank or whole-line comments. A test, lockfile, docstring, trailing comment or deleted source file fails it, naming the file and pointing at quick-fix. - `docs.check` (scripted, `python -m osf.pipeline.docs check`): relative Markdown links and backticked `path:N[-M]` on lines the run *added* must resolve; old broken links are ignored, URLs aren't fetched. Report in `.osf/docs-check.json`. - Offered in the New Run wizard wherever quick-fix is; `osf:docs` starts one. The label must be created by hand on each repository, like `osf:quick-fix`. - Specs: new `docs-pipeline`; `named-pipelines` and `new-run` updated. Guide updated. Full lane green (backend 1883, UI unit 445, browser 108). To retry cmoriarty/soundcheck#29 on it: redeploy osfd, create `osf:docs` on soundcheck, and label the issue. #98's fix ships in the same deploy, so the run won't inherit the old `.osf/change.json`.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#99
No description provided.