cmoriarty/trog
1
0
Fork 0
Code Issues 11 Pull requests Projects 1 Releases Packages Wiki Activity Actions

Hardware profiles: run trog on arbitrary GPU configurations #27

New issue
Closed
opened 2026-07-26 12:07:22 -04:00 by cmoriarty · 2 comments
cmoriarty commented 2026-07-26 12:07:22 -04:00
Owner
Copy link

Spun off from the #26 audit. The env-var slice landed there (SEAT_GPUS_IMAGE/_ANIMATE/_AUDIO + docs/self-hosting.md); this ticket is the real generalization: trog on hardware that isn't 4x16GB Turing.

Problem

Even with card assignment env-driven, the content of the stack assumes the reference rig:

  • Workflows are VRAM-sized: FLUX.2-dev Q6 sharded over 4 cards (DisTorch2), Wan-14B I2V over 3. A 24GB single-card box needs different graphs (unsharded FLUX.2, maybe the 5B Wan back), not just different device ids.
  • Brain roster is 64GB-aggregate-sized (brain/llama-swap.yaml); smaller rigs need a curated smaller roster, and the studio selector should stay honest about what fits.
  • Turing-specific choices are baked into recipes: fp16-not-bf16, GGUF-not-FP8, HeartMuLa-not-ACE-for-vocals. Ampere+ rigs could use better paths.
  • Stage parallelism (image exclusive, audio ∥ animate) is a consequence of the layout, already derived — but drain pacing/timeouts assume the reference speeds.

Sketch

A hardware profile = named bundle of: seat->GPU map, per-profile workflow selection, brain roster file, dtype/quant hints. Something like profiles/trogdor.yaml, profiles/single-24gb.yaml, selected by one stack env var. Workflow selection hooks exist already (pick_workflow, ANIMATE_WORKFLOW); the roster is already a single file.

Acceptance

  • A single-GPU (>=16GB) box runs trog test asset --mock AND a real one-item image drain with only stack-env changes.
  • Reference rig behavior byte-identical under the default profile.
Spun off from the #26 audit. The env-var slice landed there (`SEAT_GPUS_IMAGE/_ANIMATE/_AUDIO` + docs/self-hosting.md); this ticket is the real generalization: trog on hardware that isn't 4x16GB Turing. ## Problem Even with card assignment env-driven, the *content* of the stack assumes the reference rig: - **Workflows are VRAM-sized**: FLUX.2-dev Q6 sharded over 4 cards (DisTorch2), Wan-14B I2V over 3. A 24GB single-card box needs different graphs (unsharded FLUX.2, maybe the 5B Wan back), not just different device ids. - **Brain roster is 64GB-aggregate-sized** (`brain/llama-swap.yaml`); smaller rigs need a curated smaller roster, and the studio selector should stay honest about what fits. - **Turing-specific choices are baked into recipes**: fp16-not-bf16, GGUF-not-FP8, HeartMuLa-not-ACE-for-vocals. Ampere+ rigs could use better paths. - **Stage parallelism** (image exclusive, audio ∥ animate) is a consequence of the layout, already derived — but drain pacing/timeouts assume the reference speeds. ## Sketch A hardware profile = named bundle of: seat->GPU map, per-profile workflow selection, brain roster file, dtype/quant hints. Something like `profiles/trogdor.yaml`, `profiles/single-24gb.yaml`, selected by one stack env var. Workflow selection hooks exist already (`pick_workflow`, `ANIMATE_WORKFLOW`); the roster is already a single file. ## Acceptance - A single-GPU (>=16GB) box runs `trog test asset --mock` AND a real one-item image drain with only stack-env changes. - Reference rig behavior byte-identical under the default profile.
cmoriarty commented 2026-07-26 17:51:18 -04:00
Author
Owner
Copy link

Implemented (44d33d7) — verified on real hardware

A hardware profile is a named YAML bundle in server/trog_lib/profiles/: seat→GPU maps, per-lane workflow selection (t2i / t2i_fine / i2i / i2v), and the brain roster the rig is sized for. TROG_PROFILE selects one; layering is defaults < profile < SEAT_GPUS_* env.

Shipped profiles

  • trogdor (default) — byte-identical to the pre-profile hardcoded routing; tests assert the GPU matrix and the #20 workflow picks verbatim.
  • single-24gb — one card: Z-Image image seat, sized-down roster (27B @ 32K ctx), animate/i2i declared absent → jobs fail fast naming the profile instead of OOMing mid-render.

Brain rosters moved to brain/rosters/<name>.yaml, all baked into the image, BRAIN_ROSTER picks at runtime (trogdor content unchanged).

Acceptance evidence (live, on trogdor)

  • Single-GPU real generation: scratch orchestrator under TROG_PROFILE=single-24gb ran a real image one-off — seat container bound to DeviceIDs: ["0"] exactly, Z-Image lane, brain bracketed, 64px gold-key sprite delivered to the file server. Env-only change, no source edits.
  • Absent seat: animate job under single-24gb → immediate fail: "hardware profile 'single-24gb' has no animate seat…".
  • Reference rig unchanged: default profile equality is unit-asserted (127 tests green); compose defaults TROG_PROFILE=trogdor + BRAIN_ROSTER=trogdor — no new Portainer env needed, nothing changes for development or deployment.
  • New brain image boots GPU-less and serves the trogdor roster through the env-selectable entrypoint; both rosters present in-image.
  • Bonus live proof: stopping the scratch orchestrator restored the brain via the #26 shutdown fix.

Hand-off

Images are already built on the GPU host — one make deploy ships it (behavior identical under defaults). docs/self-hosting.md documents profiles + how to add one; CLAUDE.md updated for agents.

# Implemented (44d33d7) — verified on real hardware A hardware profile is a named YAML bundle in `server/trog_lib/profiles/`: seat→GPU maps, per-lane workflow selection (t2i / t2i_fine / i2i / i2v), and the brain roster the rig is sized for. `TROG_PROFILE` selects one; layering is **defaults < profile < `SEAT_GPUS_*` env**. ## Shipped profiles - **trogdor** (default) — byte-identical to the pre-profile hardcoded routing; tests assert the GPU matrix and the #20 workflow picks verbatim. - **single-24gb** — one card: Z-Image image seat, sized-down roster (27B @ 32K ctx), animate/i2i declared absent → jobs **fail fast naming the profile** instead of OOMing mid-render. Brain rosters moved to `brain/rosters/<name>.yaml`, all baked into the image, `BRAIN_ROSTER` picks at runtime (trogdor content unchanged). ## Acceptance evidence (live, on trogdor) - **Single-GPU real generation**: scratch orchestrator under `TROG_PROFILE=single-24gb` ran a real image one-off — seat container bound to `DeviceIDs: ["0"]` exactly, Z-Image lane, brain bracketed, 64px gold-key sprite delivered to the file server. Env-only change, no source edits. - **Absent seat**: animate job under single-24gb → immediate fail: "hardware profile 'single-24gb' has no animate seat…". - **Reference rig unchanged**: default profile equality is unit-asserted (127 tests green); compose defaults `TROG_PROFILE=trogdor` + `BRAIN_ROSTER=trogdor` — **no new Portainer env needed, nothing changes for development or deployment**. - New brain image boots GPU-less and serves the trogdor roster through the env-selectable entrypoint; both rosters present in-image. - Bonus live proof: stopping the scratch orchestrator restored the brain via the #26 shutdown fix. ## Hand-off Images are already built on the GPU host — one `make deploy` ships it (behavior identical under defaults). docs/self-hosting.md documents profiles + how to add one; CLAUDE.md updated for agents.
cmoriarty commented 2026-07-26 18:02:10 -04:00
Author
Owner
Copy link

CI green on HEAD (1d45a59) — pytest + studio jobs both pass with the profiles code. Closing per user direction; make deploy ships it whenever convenient (identical behavior under the trogdor defaults).

CI green on HEAD (1d45a59) — pytest + studio jobs both pass with the profiles code. Closing per user direction; `make deploy` ships it whenever convenient (identical behavior under the trogdor defaults).
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
feat/11-walking-skeleton
No results found.
Labels
Clear labels   
bug
Something is not working

  
duplicate
This issue or pull request already exists

  
enhancement
New feature

  
help wanted
Need some help

  
invalid
Something is wrong

  
question
More information is needed

  
wontfix
This won't be fixed

No labels
bug
duplicate
enhancement
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
cmoriarty/trog#27
No description provided.