cmoriarty/trog

Fork 0

05 · Asset pipeline: spec document, `generate_assets` MCP tool, model-swap conductor #4

New issue

Closed

opened 2026-07-23 00:07:07 -04:00 by cmoriarty · 10 comments

cmoriarty commented

2026-07-23 00:07:07 -04:00

Owner

0. Research step (added 2026-07-23, before carrying the stack over): survey image generators specially designed for games before defaulting to the agentic-game-dev zoo. Focus: retro-pixel style fidelity (true pixel grid, palette control) and sprite animation — walk cycles / multi-frame sheets, not single stills. Context: user bake-offs have FLUX.2 winning general image quality, but no game to be proud of yet — the gap may be game-fit, not raw quality. Current animate seat (Wan 2.2 TI2V-5B i2v) is not sprite-aware. Evaluation bar as ever: open weights, self-hosted on the 4-GPU host, swap-friendly.

All asset generation is model-swapped — no CPU-bound zoo.

Standard asset-spec document the brain writes, covering: image, sfx, music, sprite, animated sprite
generate_assets MCP tool — parses the spec into per-kind manifests and kicks off the swap process
Conductor — drain the brain LLM, load the right generator service per asset kind, process the manifest, restore the brain

Reuse the latest agentic-game-dev zoo iteration (v4 M4 batch profiles: FLUX.2 image, audio seats, animate). Key references in /Users/cmoriarty/repos/agentic-game-dev:

scripts/batch-conductor.sh — drain/swap/restore logic (incl. brain-down-across-whole-drain restructure and container-name ops fixes)
studio/studio/assets.py — spec → manifest parsing
comfyui/workflows/flux2-t2i.json etc., scripts/fetch-batch-models.sh

Local CC0 asset library: skipped for v1.

**0. Research step (added 2026-07-23, before carrying the stack over):** survey image generators *specially designed for games* before defaulting to the agentic-game-dev zoo. Focus: retro-pixel style fidelity (true pixel grid, palette control) and **sprite animation** — walk cycles / multi-frame sheets, not single stills. Context: user bake-offs have FLUX.2 winning general image quality, but no game to be proud of yet — the gap may be game-fit, not raw quality. Current animate seat (Wan 2.2 TI2V-5B i2v) is not sprite-aware. Evaluation bar as ever: open weights, self-hosted on the 4-GPU host, swap-friendly. All asset generation is model-swapped — **no CPU-bound zoo**. 1. **Standard asset-spec document** the brain writes, covering: image, sfx, music, sprite, animated sprite 2. **`generate_assets` MCP tool** — parses the spec into per-kind manifests and kicks off the swap process 3. **Conductor** — drain the brain LLM, load the right generator service per asset kind, process the manifest, restore the brain Reuse the latest agentic-game-dev zoo iteration (v4 M4 batch profiles: FLUX.2 image, audio seats, animate). Key references in `/Users/cmoriarty/repos/agentic-game-dev`: - `scripts/batch-conductor.sh` — drain/swap/restore logic (incl. brain-down-across-whole-drain restructure and container-name ops fixes) - `studio/studio/assets.py` — spec → manifest parsing - `comfyui/workflows/flux2-t2i.json` etc., `scripts/fetch-batch-models.sh` Local CC0 asset library: **skipped** for v1.

cmoriarty added this to the v1 - single user autonomous milestone

2026-07-23 00:07:07 -04:00

cmoriarty changed title from ~~Asset pipeline: spec document, generate_assets MCP tool, model-swap conductor~~ to 05 · Asset pipeline: spec document, generate_assets MCP tool, model-swap conductor

2026-07-23 00:10:04 -04:00

cmoriarty commented

2026-07-23 19:57:02 -04:00

Author

Owner

Research step done — docs/asset-gen-research.md

Short version: the game-native leaders exist but aren't self-hostable at their real quality — Retro Diffusion (dedicated sprite-sheet animation model) and PixelLab (skeleton-based animation, ships an MCP toolkit) are API businesses; their self-host offerings are generations behind. Nothing open matches them head-on.

But the open path improved since the agentic-game-dev zoo was built:

Stills: FLUX.2 stands (bake-off winner) + pixel-art LoRA + mandatory grid/palette post-pass (PixelArt-Detector / PixelOE ComfyUI nodes) — the post-pass is what makes output actually read as pixel art
Animation: community sprite-animation LoRAs for Wan 2.2 now exist (side-view walk cycles, attack sequences) — the existing animate seat becomes sprite-aware instead of generic i2v: still → i2v+LoRA → frames → post-pass → sheet + JSON manifest

Spec implications: image seat gains a pixel mode; animate seat gains sprite LoRAs; spec document gains a per-game style block (palette/resolution/outline rules every manifest inherits) — single-asset quality was never the whole "not proud of it" problem, cross-asset consistency is.

Open decision for the human, scheduled as a benchmark during implementation: generate the same briefs locally and via a few dollars of Retro Diffusion credits, compare. If local isn't close, whether asset quality justifies a paid API seat in the conductor is a deliberate bar-relaxation call — RD is per-asset cheap and PixelLab's MCP toolkit would drop into trog's architecture nearly free.

## Research step done — [docs/asset-gen-research.md](https://forgejo.underthere.xyz/cmoriarty/trog/src/branch/main/docs/asset-gen-research.md) **Short version:** the game-native leaders exist but aren't self-hostable at their real quality — Retro Diffusion (dedicated sprite-sheet animation model) and PixelLab (skeleton-based animation, ships an MCP toolkit) are API businesses; their self-host offerings are generations behind. Nothing open matches them head-on. **But the open path improved since the agentic-game-dev zoo was built:** - Stills: FLUX.2 stands (bake-off winner) + pixel-art LoRA + **mandatory grid/palette post-pass** (PixelArt-Detector / PixelOE ComfyUI nodes) — the post-pass is what makes output actually read as pixel art - **Animation: community sprite-animation LoRAs for Wan 2.2 now exist** (side-view walk cycles, attack sequences) — the existing animate seat becomes sprite-aware instead of generic i2v: still → i2v+LoRA → frames → post-pass → sheet + JSON manifest **Spec implications:** image seat gains a pixel mode; animate seat gains sprite LoRAs; spec document gains a per-game style block (palette/resolution/outline rules every manifest inherits) — single-asset quality was never the whole "not proud of it" problem, cross-asset consistency is. **Open decision for the human, scheduled as a benchmark during implementation:** generate the same briefs locally and via a few dollars of Retro Diffusion credits, compare. If local isn't close, whether asset quality justifies a paid API seat in the conductor is a deliberate bar-relaxation call — RD is per-asset cheap and PixelLab's MCP toolkit would drop into trog's architecture nearly free.

cmoriarty commented

2026-07-23 20:18:58 -04:00

Author

Owner

Implementation landed (`2d58772`) — open-source stack only, per decision

All three ticket deliverables built, deployed, CI green (32 tests):

Spec document — server/templates/assets.md: mandatory Style block (palette/grid/outline/mood — the cross-asset-consistency fix from the research) + entries per kind. animated_sprite is first-class: Base sprite + Motion, validated against sibling entries.
generate_assets MCP tool — live on trog-mcp (:8100) with asset_status. Parses docs/08-assets.md at a ref; a doc that doesn't parse is a rejection with findings for the brain, never a guess. Queues one job per profile in Postgres (trog_lib.assetq — REST queue stayed cut per the 03 audit).
Conductor — scripts/conductor.sh (+ make assets-drain): brain down once per drain, ComfyUI seat per profile (image → audio → animate, so animation bases exist), assets committed to the game repo under assets/<kind>/ — git is truth extends to assets. Restore trap covers every exit path incl. SIGPIPE (ported from the agentic-game-dev live-caught fixes).

Seats (all open weights, all already on the RAID): FLUX.2-dev Q6 sharded + core-node pixel post-pass (nearest downscale to grid + palette quantize — no custom nodes); Wan 2.2 TI2V-5B i2v → ffmpeg sprite-sheet assembly + Phaser-ready JSON manifest; ACE-Step v1 3.5B (fetched, ~7G). Workflow node names validated against the deployed ComfyUI image via /object_info (CPU-mode container, no brain disruption) — all valid.

Upgrade path staged, not active: Wan 2.2 I2V-A14B + pix3lwalk sprite-LoRA pair (FETCH_WAN14B=1 CIVITAI_TOKEN=… scripts/fetch-asset-models.sh, then a two-stage workflow + ANIMATE_WORKFLOW env). Benchmark 5B-vs-14B+LoRA when first real assets flow.

Not yet exercised: a live drain — needs the brain down ~10-30 min, so it's a deliberate run, not a side effect: write a small docs/08-assets.md in a test repo, call generate_assets, make assets-drain, review what lands in assets/. That's the next action on this ticket.

## Implementation landed (2d58772) — open-source stack only, per decision All three ticket deliverables built, deployed, CI green (32 tests): 1. **Spec document** — `server/templates/assets.md`: mandatory **Style block** (palette/grid/outline/mood — the cross-asset-consistency fix from the research) + entries per kind. `animated_sprite` is first-class: **Base** sprite + **Motion**, validated against sibling entries. 2. **`generate_assets` MCP tool** — live on trog-mcp (:8100) with `asset_status`. Parses docs/08-assets.md at a ref; a doc that doesn't parse is a rejection with findings for the brain, never a guess. Queues one job per profile in Postgres (`trog_lib.assetq` — REST queue stayed cut per the 03 audit). 3. **Conductor** — `scripts/conductor.sh` (+ `make assets-drain`): brain down once per drain, ComfyUI seat per profile (image → audio → animate, so animation bases exist), assets **committed to the game repo** under `assets/<kind>/` — git is truth extends to assets. Restore trap covers every exit path incl. SIGPIPE (ported from the agentic-game-dev live-caught fixes). **Seats (all open weights, all already on the RAID):** FLUX.2-dev Q6 sharded + core-node pixel post-pass (nearest downscale to grid + palette quantize — no custom nodes); Wan 2.2 TI2V-5B i2v → ffmpeg sprite-sheet assembly + Phaser-ready JSON manifest; ACE-Step v1 3.5B (fetched, ~7G). Workflow node names validated against the deployed ComfyUI image via /object_info (CPU-mode container, no brain disruption) — all valid. **Upgrade path staged, not active:** Wan 2.2 I2V-A14B + pix3lwalk sprite-LoRA pair (`FETCH_WAN14B=1 CIVITAI_TOKEN=… scripts/fetch-asset-models.sh`, then a two-stage workflow + `ANIMATE_WORKFLOW` env). Benchmark 5B-vs-14B+LoRA when first real assets flow. **Not yet exercised: a live drain** — needs the brain down ~10-30 min, so it's a deliberate run, not a side effect: write a small docs/08-assets.md in a test repo, call `generate_assets`, `make assets-drain`, review what lands in assets/. That's the next action on this ticket.

cmoriarty commented

2026-07-23 21:44:00 -04:00

Author

Owner

Live-drain test — step by step

Exercises all three seats (image → audio → animate) with a 3-asset spec. Brain is down ~10–30 min during step 4 — pick a quiet moment. Run everything from the trog repo checkout on the laptop.

1. Create a scratch game repo with a tiny spec

export FJ="https://forgejo.underthere.xyz/api/v1"
curl -u cmoriarty -X POST "$FJ/orgs/trog-games/repos" \
  -H 'Content-Type: application/json' \
  -d '{"name":"asset-smoke","auto_init":true,"private":false}'

Then commit the spec (paste as one block):

cat > /tmp/08-assets.md <<'EOF'
# Asset specification

## Style
- **Palette**: 12 colors, warm dusk tones — indigo, amber, moss
- **Grid**: 64x64
- **Outline**: 1px darker-shade outline, no pure black
- **Mood**: cozy nocturne, soft glow

## Assets

### lantern-keeper (sprite)
- **Brief**: small round-bodied lantern keeper holding a glowing staff, side view, clear silhouette
- **Size/format**: 64x64

### lantern-keeper-walk (animated_sprite)
- **Brief**: side-view walk cycle of the lantern keeper
- **Base**: lantern-keeper
- **Motion**: gentle walk, staff bobbing, cloak swaying, looping
- **Size/format**: 64x64, 8 frames

### soft-chime (sfx)
- **Brief**: soft warm bell chime, single strike, gentle decay
- **Size/format**: 1 sec
EOF
curl -u cmoriarty -X POST "$FJ/repos/trog-games/asset-smoke/contents/docs/08-assets.md" \
  -H 'Content-Type: application/json' \
  -d "{\"content\":\"$(base64 < /tmp/08-assets.md | tr -d '\n')\",\"message\":\"asset spec for drain smoke test\"}"

2. Queue the jobs (the real MCP path, same call the brain makes)

DOCKER_HOST=ssh://cmoriarty@trogdor docker exec trog-aegra python -c "
import asyncio, json
from trog_lib import assets, assetq, forgejo
async def go():
    text = await forgejo.read_file('trog-games/asset-smoke', assets.ASSETS_DOC)
    style, entries = assets.parse(text)
    jobs = await assetq.submit('trog-games/asset-smoke', assets.build_manifests(style, entries))
    print(json.dumps(jobs, indent=2))
asyncio.run(go())"

Expect three jobs: image (1 item), audio (1), animate (1) — in that drain order.

3. Sanity-check the queue

DOCKER_HOST=ssh://cmoriarty@trogdor docker exec trog-aegra python -m trog_lib.assetq status trog-games/asset-smoke

4. Drain (brain goes down here)

make assets-drain

Watch the log lines: brain stop → image seat up on :8189 → item 0 → seat swap → audio → swap → animate (fetches the committed sprite as its base frame) → brain restored + warmed. Every exit path restores the brain — if it dies mid-drain, the trap already brought the brain back; check with make health.

5. Verify the goods

Browse https://forgejo.underthere.xyz/trog-games/asset-smoke — expect assets/sprite/lantern-keeper.png, assets/animated_sprite/lantern-keeper-walk.png + .json, assets/sfx/soft-chime.flac (or .mp3 — whatever ComfyUI's SaveAudio emits)
Re-run the step-3 status command — all items done
The sheet: 512×64 px (8 frames of 64×64 in a row); the .json carries frameWidth/frames/fps for Phaser

6. Judge quality (this decides the 14B+LoRA benchmark)

Sprite: true 64×64 grid? Palette ≈12 colors? Reads at a glance?
Walk sheet: frames actually differ and loop? Character consistent across frames? (This is where 5B may fall short — if frames smear or the character drifts, that's the signal to fetch the Wan-14B + pix3lwalk pair: FETCH_WAN14B=1 CIVITAI_TOKEN=… ssh trogdor 'bash -s' < scripts/fetch-asset-models.sh)
Chime: usable as a game sfx?

7. Cleanup

curl -u cmoriarty -X DELETE "$FJ/repos/trog-games/asset-smoke"

If something fails: failed items show in step-3 output with errors; generator logs: DOCKER_HOST=ssh://cmoriarty@trogdor docker logs trog-assetgen; a workflow error names the node — cross-check comfyui/workflows/README.md. Re-running step 4 picks up only still-queued jobs.

## Live-drain test — step by step Exercises all three seats (image → audio → animate) with a 3-asset spec. Brain is down ~10–30 min during step 4 — pick a quiet moment. Run everything from the trog repo checkout on the laptop. ### 1. Create a scratch game repo with a tiny spec ```bash export FJ="https://forgejo.underthere.xyz/api/v1" curl -u cmoriarty -X POST "$FJ/orgs/trog-games/repos" \ -H 'Content-Type: application/json' \ -d '{"name":"asset-smoke","auto_init":true,"private":false}' ``` Then commit the spec (paste as one block): ```bash cat > /tmp/08-assets.md <<'EOF' # Asset specification ## Style - **Palette**: 12 colors, warm dusk tones — indigo, amber, moss - **Grid**: 64x64 - **Outline**: 1px darker-shade outline, no pure black - **Mood**: cozy nocturne, soft glow ## Assets ### lantern-keeper (sprite) - **Brief**: small round-bodied lantern keeper holding a glowing staff, side view, clear silhouette - **Size/format**: 64x64 ### lantern-keeper-walk (animated_sprite) - **Brief**: side-view walk cycle of the lantern keeper - **Base**: lantern-keeper - **Motion**: gentle walk, staff bobbing, cloak swaying, looping - **Size/format**: 64x64, 8 frames ### soft-chime (sfx) - **Brief**: soft warm bell chime, single strike, gentle decay - **Size/format**: 1 sec EOF curl -u cmoriarty -X POST "$FJ/repos/trog-games/asset-smoke/contents/docs/08-assets.md" \ -H 'Content-Type: application/json' \ -d "{\"content\":\"$(base64 < /tmp/08-assets.md | tr -d '\n')\",\"message\":\"asset spec for drain smoke test\"}" ``` ### 2. Queue the jobs (the real MCP path, same call the brain makes) ```bash DOCKER_HOST=ssh://cmoriarty@trogdor docker exec trog-aegra python -c " import asyncio, json from trog_lib import assets, assetq, forgejo async def go(): text = await forgejo.read_file('trog-games/asset-smoke', assets.ASSETS_DOC) style, entries = assets.parse(text) jobs = await assetq.submit('trog-games/asset-smoke', assets.build_manifests(style, entries)) print(json.dumps(jobs, indent=2)) asyncio.run(go())" ``` Expect three jobs: image (1 item), audio (1), animate (1) — in that drain order. ### 3. Sanity-check the queue ```bash DOCKER_HOST=ssh://cmoriarty@trogdor docker exec trog-aegra python -m trog_lib.assetq status trog-games/asset-smoke ``` ### 4. Drain (brain goes down here) ```bash make assets-drain ``` Watch the log lines: brain stop → image seat up on :8189 → item 0 → seat swap → audio → swap → animate (fetches the committed sprite as its base frame) → brain restored + warmed. Every exit path restores the brain — if it dies mid-drain, the trap already brought the brain back; check with `make health`. ### 5. Verify the goods - Browse https://forgejo.underthere.xyz/trog-games/asset-smoke — expect `assets/sprite/lantern-keeper.png`, `assets/animated_sprite/lantern-keeper-walk.png` + `.json`, `assets/sfx/soft-chime.flac` (or .mp3 — whatever ComfyUI's SaveAudio emits) - Re-run the step-3 status command — all items `done` - The sheet: 512×64 px (8 frames of 64×64 in a row); the `.json` carries frameWidth/frames/fps for Phaser ### 6. Judge quality (this decides the 14B+LoRA benchmark) - Sprite: true 64×64 grid? Palette ≈12 colors? Reads at a glance? - Walk sheet: frames actually differ and loop? Character consistent across frames? (This is where 5B may fall short — if frames smear or the character drifts, that's the signal to fetch the Wan-14B + pix3lwalk pair: `FETCH_WAN14B=1 CIVITAI_TOKEN=… ssh trogdor 'bash -s' < scripts/fetch-asset-models.sh`) - Chime: usable as a game sfx? ### 7. Cleanup ```bash curl -u cmoriarty -X DELETE "$FJ/repos/trog-games/asset-smoke" ``` **If something fails:** failed items show in step-3 output with errors; generator logs: `DOCKER_HOST=ssh://cmoriarty@trogdor docker logs trog-assetgen`; a workflow error names the node — cross-check comfyui/workflows/README.md. Re-running step 4 picks up only still-queued jobs.

cmoriarty commented

2026-07-23 22:02:09 -04:00

Author

Owner

Supersedes the step-by-step above — the system now does it itself

The manual guide proved too fiddly (and its step 2 misfired in practice). Commit 47bd-series adds the assets phase to the production graph (production → assets → alpha), so the flow is now just prompting the system:

For every new game run: nothing to do. The brain writes docs/08-assets.md from its own GDD/slice/build docs during the run (parser gates it — a bad spec routes findings back to the brain, bounded retries), and generation jobs land on the queue automatically.

Then, whenever the GPUs have a quiet stretch:

make assets-drain

Brain down once → image → audio → animate seats → assets committed to the game repo under assets/<kind>/ → brain restored.

Live demo ran tonight against the firefly-orchestra game (trog-games/a-tiny-browser-game-2607232205): the brain wrote a coherent spec on its first attempt — real hex palette, per-class grids (16x16 fireflies / 800x600 canvas), style-aware ("no outlines, soft radial gradients") — commit 1d17308e, 14 items queued (7 image, 6 audio, 1 animate). Drain in progress; results follow.

The scratch asset-smoke repo from the manual attempt was deleted; the queue was clean (the misfired step-2 command never reached the database).

## Supersedes the step-by-step above — the system now does it itself The manual guide proved too fiddly (and its step 2 misfired in practice). Commit 47bd-series adds the **assets phase to the production graph** (`production → assets → alpha`), so the flow is now just prompting the system: **For every new game run:** nothing to do. The brain writes `docs/08-assets.md` from its own GDD/slice/build docs during the run (parser gates it — a bad spec routes findings back to the brain, bounded retries), and generation jobs land on the queue automatically. **Then, whenever the GPUs have a quiet stretch:** ``` make assets-drain ``` Brain down once → image → audio → animate seats → assets committed to the game repo under `assets/<kind>/` → brain restored. **Live demo ran tonight** against the firefly-orchestra game (trog-games/a-tiny-browser-game-2607232205): the brain wrote a coherent spec on its first attempt — real hex palette, per-class grids (16x16 fireflies / 800x600 canvas), style-aware ("no outlines, soft radial gradients") — commit 1d17308e, 14 items queued (7 image, 6 audio, 1 animate). Drain in progress; results follow. The scratch asset-smoke repo from the manual attempt was deleted; the queue was clean (the misfired step-2 command never reached the database).

cmoriarty commented

2026-07-23 22:33:58 -04:00

Author

Owner

Live drains complete — pipeline validated end to end, conductor v2 shipped

Drain 1 (serial v1, 25.5 min): image 7/7 ✓, animate 1/1 ✓ (sheet + Phaser manifest), audio 2/6. The 4 failures were all sub-second sfx — ACE-Step rounds <1s to a zero-length latent and 400s. Fixed with duration floors (sfx ≥1s, music ≥5s), live-caught → tested → deployed.

Drain 2 (parallel v2, 2 min): the 4 requeued sfx, 4/4 ✓ through the new lane machinery — stage A (image, all-GPU, exclusive) skipped empty; stage B ran audio (device=1, :8190) and animate (device=0) lanes concurrently with per-lane in-flight tracking; brain down once, restored healthy. Conductor v2 is now the shipping conductor.

Firefly game asset set: 14/14 committed to trog-games/a-tiny-browser-game-2607232205 under assets/ — the full brain-written spec realized.

Quality read (visual review):

night-clearing background: genuinely good — palette, mood, composition all on-spec
sprites/ui: usable; pixel post-pass doing its job
animated sheet: the weak seat — frames murky. This is the predicted 5B limitation → the Wan-14B + pix3lwalk LoRA benchmark (staged in fetch-asset-models.sh) is now clearly worth running. Needs CIVITAI_TOKEN for the LoRA pair.
sfx at the 1s floor: generated clean; audition them for game-feel

Follow-ups: #14 (asset file service for intermediates/preview/library — planning), animate-seat benchmark above.

## Live drains complete — pipeline validated end to end, conductor v2 shipped **Drain 1 (serial v1, 25.5 min):** image 7/7 ✓, animate 1/1 ✓ (sheet + Phaser manifest), audio 2/6. The 4 failures were all sub-second sfx — ACE-Step rounds <1s to a zero-length latent and 400s. Fixed with duration floors (sfx ≥1s, music ≥5s), live-caught → tested → deployed. **Drain 2 (parallel v2, 2 min):** the 4 requeued sfx, 4/4 ✓ through the new lane machinery — stage A (image, all-GPU, exclusive) skipped empty; stage B ran audio (device=1, :8190) and animate (device=0) lanes concurrently with per-lane in-flight tracking; brain down once, restored healthy. Conductor v2 is now the shipping conductor. **Firefly game asset set: 14/14 committed** to trog-games/a-tiny-browser-game-2607232205 under assets/ — the full brain-written spec realized. **Quality read (visual review):** - night-clearing background: genuinely good — palette, mood, composition all on-spec - sprites/ui: usable; pixel post-pass doing its job - **animated sheet: the weak seat** — frames murky. This is the predicted 5B limitation → the Wan-14B + pix3lwalk LoRA benchmark (staged in fetch-asset-models.sh) is now clearly worth running. Needs CIVITAI_TOKEN for the LoRA pair. - sfx at the 1s floor: generated clean; audition them for game-feel **Follow-ups:** #14 (asset file service for intermediates/preview/library — planning), animate-seat benchmark above.

cmoriarty commented

2026-07-23 23:20:18 -04:00

Author

Owner

Animate benchmark done — 14B + pix3lwalk wins, now the default seat

Protocol: one 32×32 side-view character from the image seat, animated by both candidates from the same base (trog-games/animate-bench).

5B (previous seat): character identity collapses across the row — palette drifts, faces white out, stray artifacts. Confirms the firefly-pulse weakness wasn't subject-specific.
14B + pix3lwalk LoRA: identity and palette hold across all 8 frames, legs actually step. Decisive.

Changes shipped:

wan22-14b-sprite-i2v.json is the animate default; 5B stays a documented single-card fallback (ANIMATE_WORKFLOW= + SEAT_GPUS=)
Re-sharded across 3 container-local GPUs so the stage-B audio lane keeps host GPU 1 — v2 parallelism intact (conductor passes device=0,2,3)
Trigger word pix3lwalk injected by the animate profile whenever the 14B workflow runs — production graphs need no knowledge of it
Two live-caught fixes along the way, both documented in-code: I2V-A14B pairs with the Wan-2.1 VAE + WanImageToVideo (the 2.2 VAE is 5B-only; mismatch = tensor error), and civitai publishes no low-noise LoRA for this model (normal — motion is decided high-noise)
Sheet manifests now derive frame counts from the PNG itself (earlier fix) — both benchmark sheets landed at a true 8 frames

Next real drain exercises the 14B path inside the stage-B lane (the benchmark ran it as a one-off all-GPU seat).

## Animate benchmark done — 14B + pix3lwalk wins, now the default seat Protocol: one 32×32 side-view character from the image seat, animated by both candidates from the same base (`trog-games/animate-bench`). - **5B (previous seat):** character identity collapses across the row — palette drifts, faces white out, stray artifacts. Confirms the firefly-pulse weakness wasn't subject-specific. - **14B + pix3lwalk LoRA:** identity and palette hold across all 8 frames, legs actually step. Decisive. **Changes shipped:** - `wan22-14b-sprite-i2v.json` is the animate default; 5B stays a documented single-card fallback (`ANIMATE_WORKFLOW=` + `SEAT_GPUS=`) - Re-sharded across 3 container-local GPUs so the stage-B audio lane keeps host GPU 1 — v2 parallelism intact (conductor passes `device=0,2,3`) - Trigger word `pix3lwalk` injected by the animate profile whenever the 14B workflow runs — production graphs need no knowledge of it - Two live-caught fixes along the way, both documented in-code: I2V-A14B pairs with the **Wan-2.1 VAE** + `WanImageToVideo` (the 2.2 VAE is 5B-only; mismatch = tensor error), and civitai publishes no low-noise LoRA for this model (normal — motion is decided high-noise) - Sheet manifests now derive frame counts from the PNG itself (earlier fix) — both benchmark sheets landed at a true 8 frames Next real drain exercises the 14B path inside the stage-B lane (the benchmark ran it as a one-off all-GPU seat).

cmoriarty commented

2026-07-23 23:34:25 -04:00

Author

Owner

Stage-B parallel + 14B-in-lane validated live — ticket complete end to end

Validation drain (4.5 min total, brain down 03:28→03:33):

True concurrency confirmed: audio seat (GPU 1, :8190) and animate seat (14B Q6 sharded on GPUs 0,2,3, :8189) ran simultaneously — interleaved item logs, both containers up within the same second, clean lane join, brain restored healthy
Quant bumped to Q6_K first (house rule; Wan 2.2 confirmed still the open-weights ceiling — 2.5 never open-sourced); superseded Q5 pair removed from the RAID
firefly-pulse regenerated through the new seat: 4 clean pulse frames (swell → settle), palette locked to the style block — versus the original murky 5B sheet. And 4 frames is correct: the spec asks for "16x16, 4 frames"; the pixel-derived manifest now agrees with both the spec and the sheet
ambient-drone regenerated clean in parallel

Conductor v2 + 14B default + Q6 is now the fully-validated shipping configuration. animate-bench evidence repo cleaned up per review.

## Stage-B parallel + 14B-in-lane validated live — ticket complete end to end Validation drain (4.5 min total, brain down 03:28→03:33): - **True concurrency confirmed**: audio seat (GPU 1, :8190) and animate seat (14B Q6 sharded on GPUs 0,2,3, :8189) ran simultaneously — interleaved item logs, both containers up within the same second, clean lane join, brain restored healthy - **Quant bumped to Q6_K first** (house rule; Wan 2.2 confirmed still the open-weights ceiling — 2.5 never open-sourced); superseded Q5 pair removed from the RAID - **firefly-pulse regenerated through the new seat**: 4 clean pulse frames (swell → settle), palette locked to the style block — versus the original murky 5B sheet. And 4 frames is *correct*: the spec asks for "16x16, 4 frames"; the pixel-derived manifest now agrees with both the spec and the sheet - ambient-drone regenerated clean in parallel Conductor v2 + 14B default + Q6 is now the fully-validated shipping configuration. animate-bench evidence repo cleaned up per review.

cmoriarty commented

2026-07-24 01:48:51 -04:00

Author

Owner

Full system test — clean sweep (2026-07-24)

Brief → shipped game plan → self-queued assets → parallel drain → auto review page, no human hands between.

Run (mole-postman brief, thread 7460f063, 32.7 min): fun gate bounced GDD v1 → passed v2; assets phase ran in-graph for the first time — wrote a 5.4k-char spec (16-color earth palette, 48×48 grid, 7 animated sprites with proper Base refs) and queued 27 items itself.

Drain (refactored conductor, 57 min, 27/27 items, zero failures):

Stage A: 11 images, 31 min (FLUX.2 all-GPU)
Stage B parallel: 9 audio in 3 min (GPU 1) ∥ 7 animate in 25 min (14B Q6 on GPUs 0,2,3) — lanes joined clean, brain restored healthy
Post-drain hook committed assets/review.html automatically (animated sheets via steps(), audio players, briefs as context)

Every piece from this ticket's arc validated under real load in one pass: in-graph spec + parse gate, Postgres queue, parse-once conductor, wait_brain_idle, _seat.sh profiles, 14B+pix3lwalk at scale, pixel post-pass, sheet manifests from pixels, review generation. Game: trog-games/a-tiny-browser-game-2607240416.

Ticket 05 deliverables are all live and system-tested.

## Full system test — clean sweep (2026-07-24) Brief → shipped game plan → self-queued assets → parallel drain → auto review page, no human hands between. **Run** (mole-postman brief, thread 7460f063, 32.7 min): fun gate bounced GDD v1 → passed v2; **assets phase ran in-graph for the first time** — wrote a 5.4k-char spec (16-color earth palette, 48×48 grid, 7 animated sprites with proper Base refs) and queued 27 items itself. **Drain** (refactored conductor, 57 min, **27/27 items, zero failures**): - Stage A: 11 images, 31 min (FLUX.2 all-GPU) - Stage B parallel: 9 audio in 3 min (GPU 1) ∥ 7 animate in 25 min (14B Q6 on GPUs 0,2,3) — lanes joined clean, brain restored healthy - Post-drain hook committed `assets/review.html` automatically (animated sheets via steps(), audio players, briefs as context) Every piece from this ticket's arc validated under real load in one pass: in-graph spec + parse gate, Postgres queue, parse-once conductor, wait_brain_idle, `_seat.sh` profiles, 14B+pix3lwalk at scale, pixel post-pass, sheet manifests from pixels, review generation. Game: trog-games/a-tiny-browser-game-2607240416. Ticket 05 deliverables are all live and system-tested.

cmoriarty commented

2026-07-24 09:50:20 -04:00

Author

Owner

Bench drain + audio research adopted

Bench (trog-games/asset-bench, 8/8, ~10 min): the judgeable-quality spec works — the 96px knight is instantly assessable (strong silhouette, palette held, clean outline) and its 8-frame walk cycle holds character identity throughout. First live run of the kind-routed audio seats: 3 clips in 90s (footstep + sword-draw through stable-audio, jingle through ACE). Human listening verdict pending; review page auto-committed.

Audio research (docs/audio-gen-research.md): seats confirmed best-deployable — TangoFlux outbenchmarks stable-audio-open but is non-commercial (excluded; games get published). Adopted craft, parser-enforced and deployed: audio style block (Instrumentation/Tempo-key/Sound character on every audio prompt), standing anti-music sfx suffix, Variations: N takes for high-frequency sounds, Loop: yes seamless-loop post-process. Future notes: HunyuanVideo-Foley (foley synced to our animate clips), HeartMuLa (vocal themes), adaptive layering (ticket 10+).

## Bench drain + audio research adopted **Bench (trog-games/asset-bench, 8/8, ~10 min):** the judgeable-quality spec works — the 96px knight is instantly assessable (strong silhouette, palette held, clean outline) and its 8-frame walk cycle holds character identity throughout. First live run of the kind-routed audio seats: 3 clips in 90s (footstep + sword-draw through stable-audio, jingle through ACE). Human listening verdict pending; review page auto-committed. **Audio research ([docs/audio-gen-research.md](https://forgejo.underthere.xyz/cmoriarty/trog/src/branch/main/docs/audio-gen-research.md)):** seats confirmed best-deployable — TangoFlux outbenchmarks stable-audio-open but is non-commercial (excluded; games get published). Adopted craft, parser-enforced and deployed: audio style block (Instrumentation/Tempo-key/Sound character on every audio prompt), standing anti-music sfx suffix, `Variations: N` takes for high-frequency sounds, `Loop: yes` seamless-loop post-process. Future notes: HunyuanVideo-Foley (foley synced to our animate clips), HeartMuLa (vocal themes), adaptive layering (ticket 10+).

cmoriarty commented

2026-07-24 09:57:10 -04:00

Author

Owner

Closing — asset pipeline shipped, system-tested, and taught by its failures

All three deliverables live, plus everything the live runs taught us:

What shipped (the arc, newest first: 18dac25 back to 2d58772):

Spec document with mandatory Style block — visual (palette/grid/outline/mood) AND audio (instrumentation/tempo-key/sound character); Variations: takes and Loop: seamless loops; animated_sprite first-class with Base+Motion. Parser rejects with findings, never guesses; the assets phase runs in-graph with the parser as its gate.
generate_assets + asset_status on trog-mcp; Postgres queue (trog_lib.assetq).
Conductor v2: brain down once per drain, stage A image exclusive (FLUX.2 sharded + core-node pixel post-pass), stage B parallel lanes — kind-routed audio seats (ACE music :8095, stable-audio sfx :8096) beside the Wan-14B+pix3lwalk animate seat (Q6, 3-GPU shard). Assets commit to the game repo — git is truth includes assets. Auto-generated review page per drain.
Research docs (asset-gen, audio-gen) with adopt/reject reasoning; make asset-bench for judgeable quality checks; make try-updates for tested dependency bumps (born from this ticket's cadence).

What the live runs caught (each fixed same-day, most with a regression test): ACE 400s on sub-second latents; sheet manifests lying about frame counts (now pixel-derived); 5B animate losing character identity (benchmarked out, cut — no below-bar fallbacks); sfx-as-songs from music-model routing; the 14B/2.1-VAE pairing; brain_idle not actually waiting.

Final numbers: last full system test ran brief → shipped plan → self-queued 27 items → 27/27 drained → review page, hands-free. Bench: 8/8 with an instantly-judgeable knight + walk cycle.

Handed to follow-ups: #14 (file service — serves the review pages), #15 (asset critics — the judge that would have caught the sfx failure), #16 (audio-gen investigations). Test repos and local run artifacts cleaned.

The conductor drains, the ledger remembers, the reviews render. On to the studio.

## Closing — asset pipeline shipped, system-tested, and taught by its failures All three deliverables live, plus everything the live runs taught us: **What shipped** (the arc, newest first: 18dac25 back to 2d58772): - **Spec document** with mandatory Style block — visual (palette/grid/outline/mood) AND audio (instrumentation/tempo-key/sound character); `Variations:` takes and `Loop:` seamless loops; animated_sprite first-class with Base+Motion. Parser rejects with findings, never guesses; the assets phase runs in-graph with the parser as its gate. - **generate_assets + asset_status** on trog-mcp; Postgres queue (trog_lib.assetq). - **Conductor v2**: brain down once per drain, stage A image exclusive (FLUX.2 sharded + core-node pixel post-pass), stage B parallel lanes — kind-routed audio seats (ACE music :8095, stable-audio sfx :8096) beside the Wan-14B+pix3lwalk animate seat (Q6, 3-GPU shard). Assets commit to the game repo — git is truth includes assets. Auto-generated review page per drain. - **Research docs** (asset-gen, audio-gen) with adopt/reject reasoning; `make asset-bench` for judgeable quality checks; `make try-updates` for tested dependency bumps (born from this ticket's cadence). **What the live runs caught** (each fixed same-day, most with a regression test): ACE 400s on sub-second latents; sheet manifests lying about frame counts (now pixel-derived); 5B animate losing character identity (benchmarked out, cut — no below-bar fallbacks); sfx-as-songs from music-model routing; the 14B/2.1-VAE pairing; brain_idle not actually waiting. **Final numbers:** last full system test ran brief → shipped plan → self-queued 27 items → 27/27 drained → review page, hands-free. Bench: 8/8 with an instantly-judgeable knight + walk cycle. **Handed to follow-ups:** #14 (file service — serves the review pages), #15 (asset critics — the judge that would have caught the sfx failure), #16 (audio-gen investigations). Test repos and local run artifacts cleaned. The conductor drains, the ledger remembers, the reviews render. On to the studio.

cmoriarty closed this issue

2026-07-24 09:57:11 -04:00