Scripted steps and git-heavy tests pay a Python start-up and a fresh repository each; cut both #230

Closed
opened 2026-10-20 19:23:16 -04:00 by cmoriarty · 1 comment
Owner

Tier 2 of the test-suite speed-up (#227).

  1. Every scripted step runs $OSF_PYTHON -m osf.pipeline.<module>, and importing osf.pipeline.fixloop costs 377 ms, of which 254 ms is osf.engine.executors pulling in httpx through osf.pipeline.loader. A quick-fix run has about ten such steps, so 2 to 4 s of a 20 s simulated run is imports, and real runs pay the same. Proposal: make the pipeline modules' command-line entry points import only what they use (lazy imports of the engine and the agent client in osf.pipeline.loader, or a lighter module for the step commands), measured with python -X importtime.

  2. tests/unit/test_parallel_groups.py, test_parallel_git.py and test_scheduler.py build a git repository and a database per test; the 25 slowest backend tests are theirs (15 s down to 2 s each). Proposal: build the repository once per session as a template and copy it per test (git clone --shared or a plain copy of the directory), and see what else their set-up does per test that could be shared. Measure the backend step before and after (131 s with worksteal on 2026-10-20).

Tier 2 of the test-suite speed-up (#227). 1. Every scripted step runs `$OSF_PYTHON -m osf.pipeline.<module>`, and importing osf.pipeline.fixloop costs 377 ms, of which 254 ms is osf.engine.executors pulling in httpx through osf.pipeline.loader. A quick-fix run has about ten such steps, so 2 to 4 s of a 20 s simulated run is imports, and real runs pay the same. Proposal: make the pipeline modules' command-line entry points import only what they use (lazy imports of the engine and the agent client in osf.pipeline.loader, or a lighter module for the step commands), measured with `python -X importtime`. 2. tests/unit/test_parallel_groups.py, test_parallel_git.py and test_scheduler.py build a git repository and a database per test; the 25 slowest backend tests are theirs (15 s down to 2 s each). Proposal: build the repository once per session as a template and copy it per test (`git clone --shared` or a plain copy of the directory), and see what else their set-up does per test that could be shared. Measure the backend step before and after (131 s with worksteal on 2026-10-20).
Author
Owner

Shipped (29eaee0), the first half; the second half is closed on a measurement.

  1. osf.pipeline and osf.engine resolve the loader's, the executors' and the scheduler's names on first use, and osf.forgejo and the gitignore step import httpx where the client is opened. A scripted step's python -m osf.pipeline.<module> now loads neither the engine nor httpx (tests/unit/test_import_cost.py checks it in a fresh interpreter). With warm caches: fixloop 128 ms to 65, apply 116 to 62, gitflow 115 to 63, deps 116 to 69, review 111 to 81; the 377 ms first reading was cold and under load, which is what a step meets in a busy run. quick-fix alone: 21.4 s in the morning, 14.0 s after (part of that is a warmer cache).

  2. The slow backend tests: test_four_ready_groups_start_three_and_the_fourth_waits_for_one ticks until five groups have started and ended, each with a real worktree, a copied Node tree and a Python environment made in the background; its set-up's git init and one commit is a fraction of a second. The time is what the test tests, so no template repository.

Shipped (`29eaee0`), the first half; the second half is closed on a measurement. 1. `osf.pipeline` and `osf.engine` resolve the loader's, the executors' and the scheduler's names on first use, and `osf.forgejo` and the gitignore step import `httpx` where the client is opened. A scripted step's `python -m osf.pipeline.<module>` now loads neither the engine nor httpx (`tests/unit/test_import_cost.py` checks it in a fresh interpreter). With warm caches: `fixloop` 128 ms to 65, `apply` 116 to 62, `gitflow` 115 to 63, `deps` 116 to 69, `review` 111 to 81; the 377 ms first reading was cold and under load, which is what a step meets in a busy run. `quick-fix` alone: 21.4 s in the morning, 14.0 s after (part of that is a warmer cache). 2. The slow backend tests: `test_four_ready_groups_start_three_and_the_fourth_waits_for_one` ticks until five groups have started and ended, each with a real worktree, a copied Node tree and a Python environment made in the background; its set-up's `git init` and one commit is a fraction of a second. The time is what the test tests, so no template repository.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#230
No description provided.