Shared caches with two runs at once, the store reused after a deploy, and slimming a finished run's environments #192
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Follow-up to #67. Lean run folders have been live since
4309dfe(2026-10-17), and were archived in OpenSpec aslean-run-foldersin874eb27.Where it stands
node_modulesis 132 MB.home/.npm,cache/andtmp/are at 0, and there is no opencode plugin folder.A finished run is now almost entirely its dependency environments.
1. Two runs installing at once (#67's task 9.6)
With the operator's go, start two quick-fix runs on one throwaway repository at once. Check that:
home/.npm/_cacacheis a link to the store, and each_npxis its own;/home/braidgains no.npmand no.cache/uv(name here anything else that appears there);tmp/holds nothing buthandoff/;2. The store reused after a deploy (task 9.7)
After the next deploy that follows 1: a run on the same repository installs its packages from the store. The store does not grow, and its
script.dependenciestakes less time than the first install in 1.3. Production checks after the deploy, not yet recorded (task 9.5)
braid-check-shared-cacheslines in the deploy log.osfd doctoron production.tmp/holds nothing buthandoff/. The rest of the runs that finished before the deploy was read at archive withGET /api/runs/{id}/footprint?measure=1: runs 55 to 73 hold nohome/.npmorcache/contents (caches 0), and the succeeded ones'tmp/is empty, except run 73's 12,750 bytes, which the breakdown does not split.4. Slimming a finished run
#67's acceptance, carried here:
Why it was left (lean-run-folders design D9). A finished run's
node_modulesand.venvare now the largest part of it: 137 of 169 MB in run 67, 264 of 468 MB in the flipart runs, and 132 of 139 MB in run 74. But any finished run can be re-opened, by Resume, a redo or an accept, and a re-opened step expects the environmentsscript.dependenciesmade:script.dependencies.testsreinstalls only the directories whose manifests changed;So a safe slim needs a re-install, keyed on
.osf/deps.json, that holds a re-opened run's steps until it finishes. Until then, archiving with "delete its files" reclaims a run whole.The breakdown to measure it by (design D8) is in place:
GET /api/runs/{id}/footprint?measure=1and the archive dialog split a run into dependencies, caches, temp, sessions and work.Done when
Part 4 shipped in PR 468 (main
ddfd8a93), archived asbatch-2026-11-09b(PR 469). Parts 1 to 3 are production recordings and stay open here.What changed.
.osf/deps.jsonrecords as installed (node_modules,.venv), only when the working copy is inside the run's folder, each environment's parent inside the working copy, and pastunsafe_to_delete; a link goes as a link. Nothing else of the working copy is touched. The footprint'sdependenciespart shows what is left.$OSF_PYTHON -m osf.pipeline.deps install --root . --changedruns in the working copy with the dependencies step's own environment.--changednow counts a recorded environment the disk lacks as changed, so an unchanged manifest is installed again. A failed install is logged and the steps run anyway. The hold shows only in the log (installing backend/.venv again before its steps run, thenenvironments installed again in N s).Evidence on the laptop.
tests/engine/test_slim_finished_run.py(what the tidy lists, and what it refuses),tests/pipeline/test_deps_missing_environments.py(the record read; the--changedpass reinstalls a missing environment and runs nothing when it is there),tests/engine/test_restore_environments.py(the hold, the release, the command and its environment).slimmed-run-resumes: a quick-fix run onhello-venvis stopped after its first turn, resumed withafter: slimmed(a new operator option that waits until the footprint measures no dependencies), and merges withbackend/.venv/bin/pythonback andscript.dependenciesrun once. Its osfd log:tidied finished run …: 126635 bytes (dependencies 32955, caches 93680),installing backend/.venv again before its steps run,environments installed again in 2 s, and at the endtidied finished run …: (dependencies 11050892, caches 93680, temp 16974364)../tools/sim.sh --smoke: 7 of 7.Left for production (parts 1 to 3): two quick-fix runs at once on a throwaway repository under
braid-testing/, with the operator's go; the store reused after the next deploy; thebraid-check-shared-cacheslines,osfd doctor, and the run folders'tmp/. Plus the first real slim and re-install: a finished run's footprint readingdependencies 0, and the two log lines above on a resume.The first slim on production, at the boot of
ddfd8a9(2026-11-09, 20:41 UTC).boot: tidying 14 finished run(s)' caches and temporary files, then one line per run; 13 had environments, run 187 (stopped during propose, before its install) had nothing to delete:About 1.40 GB in all, the whole of the
dependenciespart of every finished run on the ledger; their footprints now readdependencies 0. Nothing else was touched (each line's bytes are alldependencies). The re-install on a re-open has not yet happened on production: the next resume or redo of one of these runs will showinstalling node_modules again before its steps runin osfd's log.Parts 1 to 3, read from production on 2026-11-11 (
f27905f, idle). Closing: part 4 shipped and slimmed production on 11-09 (above), and these are the recordings.3. Production checks
The deploy log of
f27905fhas all sevenbraid-check-shared-cacheslinesok:_npxnpm cache clean --forceleft the shared store's 5 filesUV_LINK_MODE=clone).venvreached neither the other run nor the cacheuv run --withof one package at onceosfd doctoron production: 0 failed, 1 warned (the sandbox, braid#203). "the shared package caches are writable" and "runs get UV_LINK_MODE=clone" are both ok.Run folders, all 19 on disk:
home/.npm/_cacacheis a link to the shared store, and no run has a file incache/;tmp/is empty; the 4 stopped runs keep theirs, as designed;/home/braidholds its three shell dotfiles and nothing else: no.npm, no.cache.2. The store reused after a deploy
Seven installs of one project (
scratch-vllm-fp8, and run 192's copy of it), 11-06 to 11-11. The last four each ran after a new deploy.npm ciinscript.dependenciesRuns 178 and 179 on
brooke: 100 packages in 3.1 and 2.7 s. Installs that take seconds come from the store.Not measured: the store's size before and after one such install. The shared cache is about 2.8 GB today (npm 2.2 GB, uv 0.6 GB), against 840 MB on 10-18. Nothing prunes it.
1. Two runs installing at once
Not run as written (two quick-fix runs started together on one repository). What there is instead:
script.dependenciesa minute apart (3.2 s and 3.7 s), both succeeded, and their folders read as in 3.No two
script.dependenciessteps have overlapped to the second in the 23 runs on the ledger.Part 4, one thing still not seen on production: a re-opened slimmed run installing again. No slimmed run has been resumed since 11-09, so that rests on the sim
slimmed-run-resumes. Today's boot tidied 19 finished runs and found nothing left to delete.An install that breaks with two runs at once, or a resume that fails to reinstall, would be a new ticket.