cmoriarty/braid
1
0
Fork 0
Code Issues 15 Pull requests Projects Releases Packages 1 Wiki Activity Actions

Support more than one pipeline graph per repository #5

New issue
Closed
opened 2026-09-13 08:16:15 -04:00 by cmoriarty · 1 comment
cmoriarty commented 2026-09-13 08:16:15 -04:00
Owner
Copy link

Today a repository has exactly one process: .osf/pipeline.yaml, read out of git at the
run's base sha (src/osf/pipeline/loader.py), or the built-in default when that file is
absent. Every run gets the same graph.

That graph hard-codes one branching model — feature branching: a worktree on
osf/run_<id> cut from main, and pr.open (open_pull_request in
src/osf/forgejo.py, base="main") pushes it and opens a PR back into main.

Want

Choose the graph per run, so the same repository can offer more than one process. The
first motivating second graph is a proper git-flow pipeline:

  • feature/* branches cut from and merged back into develop, not main
  • a release/* path: branch from develop, stabilise, merge to main + tag, back-merge to develop
  • a hotfix/* path: branch from main, merge to both main and develop

The existing graph stays as the default feature-branch pipeline.

Create a third graph built for speed called minimalist, which uses openspec lightly, just propose through archive, no explore or review. It also uses git feature branching. It also runs unit tests, and UI tests when required. Speed is the priority on this, but without compromising specs, version control and testing.

Sketch

  • Storage. e.g. .osf/pipelines/<name>.yaml, with .osf/pipeline.yaml still honoured
    as default. Every graph read out of git at the base sha, as now.
  • Selection. A pipeline name on run start (UI "new run", tools/drive.py, intake).
    For Forgejo intake, map from an issue label (e.g. osf:git-flow) with a fallback to default.
  • Recorded, not re-resolved. The chosen name (and sha) is written into the run's
    start event so recovery and redo replay against the same graph.
  • Branch parameters, not forks. Base branch, work-branch prefix and PR target become
    pipeline-level settings instead of "main" / osf/run_ literals, so git-flow is
    mostly configuration plus a few new scripted nodes (merge, tag, back-merge).
  • UI. Show the pipeline name on the run list and spine header.

Open questions

  • Is a release a run of its own graph, or a gate at the end of a feature run?
  • Merging and tagging are irreversible effects — they need ::osf:effect:: markers and
    redo semantics like pr.open has.
  • pr.merge is still commented out in the default node table; git-flow likely needs it first.

Done when

  • Two graphs coexist in one repo and a run can be started on either.
  • The run records which graph it used; boot recovery resumes on that graph.
  • A git-flow run opens its feature PR against develop.
  • default_pipeline() behaviour is unchanged for repos with a single .osf/pipeline.yaml.
Today a repository has exactly one process: `.osf/pipeline.yaml`, read out of git at the run's base sha (`src/osf/pipeline/loader.py`), or the built-in default when that file is absent. Every run gets the same graph. That graph hard-codes one branching model — **feature branching**: a worktree on `osf/run_<id>` cut from `main`, and `pr.open` (`open_pull_request` in `src/osf/forgejo.py`, `base="main"`) pushes it and opens a PR back into `main`. ## Want Choose the graph per run, so the same repository can offer more than one process. The first motivating second graph is a proper **git-flow** pipeline: - `feature/*` branches cut from and merged back into `develop`, not `main` - a `release/*` path: branch from `develop`, stabilise, merge to `main` + tag, back-merge to `develop` - a `hotfix/*` path: branch from `main`, merge to both `main` and `develop` The existing graph stays as the default feature-branch pipeline. Create a third graph built for speed called minimalist, which uses openspec lightly, just propose through archive, no explore or review. It also uses git feature branching. It also runs unit tests, and UI tests when required. Speed is the priority on this, but without compromising specs, version control and testing. ## Sketch - **Storage.** e.g. `.osf/pipelines/<name>.yaml`, with `.osf/pipeline.yaml` still honoured as `default`. Every graph read out of git at the base sha, as now. - **Selection.** A pipeline name on run start (UI "new run", `tools/drive.py`, intake). For Forgejo intake, map from an issue label (e.g. `osf:git-flow`) with a fallback to default. - **Recorded, not re-resolved.** The chosen name (and sha) is written into the run's start event so recovery and redo replay against the same graph. - **Branch parameters, not forks.** Base branch, work-branch prefix and PR target become pipeline-level settings instead of `"main"` / `osf/run_` literals, so git-flow is mostly configuration plus a few new scripted nodes (merge, tag, back-merge). - **UI.** Show the pipeline name on the run list and spine header. ## Open questions - Is a release a run of its own graph, or a gate at the end of a feature run? - Merging and tagging are irreversible effects — they need `::osf:effect::` markers and redo semantics like `pr.open` has. - `pr.merge` is still commented out in the default node table; git-flow likely needs it first. ## Done when - Two graphs coexist in one repo and a run can be started on either. - The run records which graph it used; boot recovery resumes on that graph. - A git-flow run opens its feature PR against `develop`. - `default_pipeline()` behaviour is unchanged for repos with a single `.osf/pipeline.yaml`.
cmoriarty commented 2026-09-26 12:02:02 -04:00
Author
Owner
Copy link

Shipped in b8cb11a (change archived as 2026-09-26-named-pipelines, spec openspec/specs/named-pipelines), deployed to production.

  • A run is started on a named pipeline: default = .osf/pipeline.yaml, <name> = .osf/pipelines/<name>.yaml, falling back to a built-in of that name; unknown names block admission with the name.
  • Built in: minimalist (propose → apply → unit tests, browser tests only when UI files changed → PR → merge → archive; no explore/review), git-flow (feature/* from and into develop), git-flow-release (release/* from develop → PR into main → tag vX.(Y+1).0 → merge back into develop), git-flow-hotfix (minimalist on hotfix/* from main → archive → tag vX.Y.(Z+1) → merge back into develop). Release is its own graph, not a gate on a feature run.
  • branches: {base, work_prefix, pr_target} replace the main / osf/ literals; PRs now target the branch the run was cut from.
  • New stages git.flow.release-start, git.flow.pr.open, git.flow.tag, git.flow.back-merge emit ::osf:effect:: markers and are idempotent under redo: supersede.
  • The pipeline name and base sha are recorded on admission (migration 4); recovery/redo load exactly that pair.
  • Selection: New Run picker (GET /api/pipelines), pipeline on POST /api/runs, tools/drive.py --pipeline, or an osf:<name> issue label (which also starts the run). Shown on the run list and spine header.
  • default_pipeline() is unchanged; git-flow needs a develop branch (the run blocks naming it otherwise).
Shipped in b8cb11a (change archived as `2026-09-26-named-pipelines`, spec `openspec/specs/named-pipelines`), deployed to production. - A run is started on a named pipeline: `default` = `.osf/pipeline.yaml`, `<name>` = `.osf/pipelines/<name>.yaml`, falling back to a built-in of that name; unknown names block admission with the name. - Built in: `minimalist` (propose → apply → unit tests, browser tests only when UI files changed → PR → merge → archive; no explore/review), `git-flow` (feature/* from and into develop), `git-flow-release` (release/* from develop → PR into main → tag vX.(Y+1).0 → merge back into develop), `git-flow-hotfix` (minimalist on hotfix/* from main → archive → tag vX.Y.(Z+1) → merge back into develop). Release is its own graph, not a gate on a feature run. - `branches: {base, work_prefix, pr_target}` replace the `main` / `osf/` literals; PRs now target the branch the run was cut from. - New stages `git.flow.release-start`, `git.flow.pr.open`, `git.flow.tag`, `git.flow.back-merge` emit `::osf:effect::` markers and are idempotent under `redo: supersede`. - The pipeline name and base sha are recorded on admission (migration 4); recovery/redo load exactly that pair. - Selection: New Run picker (`GET /api/pipelines`), `pipeline` on `POST /api/runs`, `tools/drive.py --pipeline`, or an `osf:<name>` issue label (which also starts the run). Shown on the run list and spine header. - `default_pipeline()` is unchanged; git-flow needs a `develop` branch (the run blocks naming it otherwise).
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
archive/prefect-kestra-experiments
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
cmoriarty/braid#5
No description provided.