Scripted steps and git-heavy tests pay a Python start-up and a fresh repository each; cut both #230
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Tier 2 of the test-suite speed-up (#227).
Every scripted step runs
$OSF_PYTHON -m osf.pipeline.<module>, and importing osf.pipeline.fixloop costs 377 ms, of which 254 ms is osf.engine.executors pulling in httpx through osf.pipeline.loader. A quick-fix run has about ten such steps, so 2 to 4 s of a 20 s simulated run is imports, and real runs pay the same. Proposal: make the pipeline modules' command-line entry points import only what they use (lazy imports of the engine and the agent client in osf.pipeline.loader, or a lighter module for the step commands), measured withpython -X importtime.tests/unit/test_parallel_groups.py, test_parallel_git.py and test_scheduler.py build a git repository and a database per test; the 25 slowest backend tests are theirs (15 s down to 2 s each). Proposal: build the repository once per session as a template and copy it per test (
git clone --sharedor a plain copy of the directory), and see what else their set-up does per test that could be shared. Measure the backend step before and after (131 s with worksteal on 2026-10-20).Shipped (
29eaee0), the first half; the second half is closed on a measurement.osf.pipelineandosf.engineresolve the loader's, the executors' and the scheduler's names on first use, andosf.forgejoand the gitignore step importhttpxwhere the client is opened. A scripted step'spython -m osf.pipeline.<module>now loads neither the engine nor httpx (tests/unit/test_import_cost.pychecks it in a fresh interpreter). With warm caches:fixloop128 ms to 65,apply116 to 62,gitflow115 to 63,deps116 to 69,review111 to 81; the 377 ms first reading was cold and under load, which is what a step meets in a busy run.quick-fixalone: 21.4 s in the morning, 14.0 s after (part of that is a warmer cache).The slow backend tests:
test_four_ready_groups_start_three_and_the_fourth_waits_for_oneticks until five groups have started and ended, each with a real worktree, a copied Node tree and a Python environment made in the background; its set-up'sgit initand one commit is a fraction of a second. The time is what the test tests, so no template repository.