cmoriarty/trog
1
0
Fork 0
Code Issues 11 Pull requests Projects 1 Releases Packages Wiki Activity Actions

05.7 · Sprite & pixel-art quality: research the craft, bench models/variants against the FLUX seat #20

New issue
Closed
opened 2026-07-24 20:41:22 -04:00 by cmoriarty · 2 comments
cmoriarty commented 2026-07-24 20:41:22 -04:00
Owner
Copy link

The fixture work (#17) put current sprite quality under a magnifying glass, and the critic's notes were fair: the knight has painterly mush instead of deliberate pixel clusters, no line discipline, a muddy plume; the apple's leaf is mangled; faces/details break on anything intricate (card pips). The seat produces recognizable sprites, not good pixel art. Investigate-then-decide, 05.3 bench pattern: research first, then candidates generate real assets side-by-side with the incumbent, judged on a review page.

Incumbent: FLUX.2-dev Q6 + core-node pixel post-pass (nearest downscale to grid + palette quantize) — per docs/asset-gen-research.md the post-pass is what makes output read as pixel art at all. Evaluation bar as ever: open weights, commercially usable license, self-hosted, swap-friendly, and it must beat the incumbent on something measurable.

Research (before any GPU time)

  • What makes good pixel art, concretely — the craft vocabulary a spec/prompt can enforce: deliberate clusters, hue-shifted ramps (not tint/shade), selective outlines (sel-out), anti-aliasing only at form edges, silhouette-first readability, cluster economy at the target grid. Sources: Pedro Medeiros / saint11 pixel-art tutorials, Lospec tutorials + palette discipline, pixelart-tutorials aggregators. Output: a craft checklist that becomes (a) prompt/style-block vocabulary and (b) critic vocabulary — teach the judge the same standards.
  • Prompt craft for the current seat — how far does the incumbent stretch with better prompting (cluster/outline/ramp vocabulary, explicit palette naming, negative prompts against gradients/noise)? Cheapest possible win; bench it first as its own variant.
  • Post-pass upgrades — the quantize step is doing heavy lifting today. Investigate: better palette extraction (k-means in OKLab?), ordered/error-diffusion dithering control, an outline/sel-out synthesis pass, cluster cleanup (isolated-pixel removal). These are CPU-cheap and model-agnostic — they compound with any model choice.
  • Models/variants to shortlist (license first, as always):
    • Pixel-art LoRAs for FLUX (keeps the seat; a LoRA swap is the cheapest model change)
    • Pixel-art LoRAs/checkpoints for SDXL-class bases (smaller, faster, arguably better LoRA ecosystem for pixel styles)
    • Retro Diffusion: previously excluded for its API/licensing — re-check whether any weights are self-hostable under a usable license
    • Anything purpose-built for sprites/tiles that is open-weight (survey Civitai/HF; most "pixel" models are style filters — the bar is grid-true clusters at 96x96, not pixel-flavored illustration)
  • Animation quality angle (stretch): pix3lwalk covers walk cycles only; card-flip failed twice on physics/continuity (#17). Survey whether any I2V LoRA/model does non-locomotion sprite motion credibly, or whether attack/flip/use animations need a different technique entirely (frame-interpolation from keyframes, AnimateDiff-style sprite pipelines).

Bench (05.3 pattern, tier-2 seat CLI does the lifting)

  • Canned spec of quality-revealing subjects at 96x96 and 32x32: character with silhouette (knight), object with fine detail (playing card / potion bottle), organic shape (apple/tree), tile (grass/brick), UI element (button). Same briefs through every variant.
  • Variants: incumbent as-is · incumbent + craft prompting · incumbent + upgraded post-pass · each shortlisted LoRA/model · combinations that survive.
  • Review page: side-by-side grid per subject (pixel-true upscale + native speck, the established pattern), metadata chips (model/LoRA, license, VRAM, gen time), critic verdicts alongside for calibration — do the critics' judgments track the eye test?
  • Eye test decides; the craft checklist is the rubric.

Decision output

Per variant: adopt (seat/workflow change wired), reject (reason in docs/asset-gen-research.md), or defer (trigger that reopens). Plus: spec template gains the craft vocabulary the research produces, and the visual critic gains the same standards so future drains are judged by them.

Working material: seat CLI + trog-games/seat-scratch for one-off generation (#17), review-page tooling from the 05.3 bench (trog-games/audio-model-bench bench/build-bench-page.py as the template), current workflow comfyui/workflows/flux2-pixel-t2i.json.

The fixture work (#17) put current sprite quality under a magnifying glass, and the critic's notes were fair: the knight has painterly mush instead of deliberate pixel clusters, no line discipline, a muddy plume; the apple's leaf is mangled; faces/details break on anything intricate (card pips). The seat produces *recognizable* sprites, not *good* pixel art. Investigate-then-decide, 05.3 bench pattern: research first, then candidates generate real assets side-by-side with the incumbent, judged on a review page. **Incumbent:** FLUX.2-dev Q6 + core-node pixel post-pass (nearest downscale to grid + palette quantize) — per docs/asset-gen-research.md the post-pass is what makes output read as pixel art at all. Evaluation bar as ever: open weights, commercially usable license, self-hosted, swap-friendly, and it must beat the incumbent on something measurable. ### Research (before any GPU time) - [ ] **What makes good pixel art, concretely** — the craft vocabulary a spec/prompt can enforce: deliberate clusters, hue-shifted ramps (not tint/shade), selective outlines (sel-out), anti-aliasing only at form edges, silhouette-first readability, cluster economy at the target grid. Sources: Pedro Medeiros / saint11 pixel-art tutorials, Lospec tutorials + palette discipline, pixelart-tutorials aggregators. Output: a craft checklist that becomes (a) prompt/style-block vocabulary and (b) critic vocabulary — teach the judge the same standards. - [ ] **Prompt craft for the current seat** — how far does the incumbent stretch with better prompting (cluster/outline/ramp vocabulary, explicit palette naming, negative prompts against gradients/noise)? Cheapest possible win; bench it first as its own variant. - [ ] **Post-pass upgrades** — the quantize step is doing heavy lifting today. Investigate: better palette extraction (k-means in OKLab?), ordered/error-diffusion dithering control, an outline/sel-out synthesis pass, cluster cleanup (isolated-pixel removal). These are CPU-cheap and model-agnostic — they compound with any model choice. - [ ] **Models/variants to shortlist** (license first, as always): - Pixel-art LoRAs for FLUX (keeps the seat; a LoRA swap is the cheapest model change) - Pixel-art LoRAs/checkpoints for SDXL-class bases (smaller, faster, arguably better LoRA ecosystem for pixel styles) - Retro Diffusion: previously excluded for its API/licensing — re-check whether any weights are self-hostable under a usable license - Anything purpose-built for sprites/tiles that is open-weight (survey Civitai/HF; most "pixel" models are style filters — the bar is grid-true clusters at 96x96, not pixel-flavored illustration) - [ ] **Animation quality angle** (stretch): pix3lwalk covers walk cycles only; card-flip failed twice on physics/continuity (#17). Survey whether any I2V LoRA/model does non-locomotion sprite motion credibly, or whether attack/flip/use animations need a different technique entirely (frame-interpolation from keyframes, AnimateDiff-style sprite pipelines). ### Bench (05.3 pattern, tier-2 seat CLI does the lifting) - [ ] Canned spec of quality-revealing subjects at 96x96 and 32x32: character with silhouette (knight), object with fine detail (playing card / potion bottle), organic shape (apple/tree), tile (grass/brick), UI element (button). Same briefs through every variant. - [ ] Variants: incumbent as-is · incumbent + craft prompting · incumbent + upgraded post-pass · each shortlisted LoRA/model · combinations that survive. - [ ] Review page: side-by-side grid per subject (pixel-true upscale + native speck, the established pattern), metadata chips (model/LoRA, license, VRAM, gen time), critic verdicts alongside for calibration — do the critics' judgments track the eye test? - [ ] Eye test decides; the craft checklist is the rubric. ### Decision output Per variant: adopt (seat/workflow change wired), reject (reason in docs/asset-gen-research.md), or defer (trigger that reopens). Plus: spec template gains the craft vocabulary the research produces, and the visual critic gains the same standards so future drains are judged by them. **Working material:** seat CLI + `trog-games/seat-scratch` for one-off generation (#17), review-page tooling from the 05.3 bench (`trog-games/audio-model-bench` `bench/build-bench-page.py` as the template), current workflow `comfyui/workflows/flux2-pixel-t2i.json`.
cmoriarty commented 2026-07-25 15:54:54 -04:00
Author
Owner
Copy link

05.7 complete — research, bench, decisions wired

Review page (eye test is yours): http://trogdor:3923/games/sprite-bench/assets/review.html — 3 models x 2 prompt styles x 2 post-passes x 5 subjects x 2 grids, fixed seeds, critic verdicts alongside. Bench kit: trog-games/sprite-bench.

Root cause of the #17 complaints

The "mangled apple leaf" was the post-pass, not the model: in-graph ImageQuantize (median cut) merged small-area hues away — the model painted the leaf green every time. The "painterly mush" was prompt-level: plain briefs render airbrushed cartoons the post-pass then has to rescue.

Adopted (deployed)

  • Craft rubric — docs/pixel-art-craft.md (saint11/Lospec vocabulary: clusters, hue-shifted ramps, sel-out, banding, pillow shading, grid budget) wired into the spec template and the visual critic. Critic enforcement is a style-block Craft: opt-in — legacy specs keep their honest bar, new games opt in via template.
  • Craft prompting — transforms the incumbent (deliberate clusters/sel-out before any post-pass), fixes Z-Image's typography spam. Caveat: sprite-phrased craft language hurts tiles; briefs stay kind-aware.
  • CPU palette post — trog_lib/pixel_post.py: OKLab k-means snapped to true source colors + orphan-pixel cleanup, replacing ImageQuantize in both flux2 graphs; applied to every PNG artifact, sheets included.
  • klein-4B + Limbicnation pixel LoRA as the sprite/ui seat — best cluster quality of the field at ~10-15s vs ~240s, Apache-2.0 base+LoRA. Klein pairs with the Qwen3-4B TE (Mistral dim-mismatches, live-caught). Backgrounds/i2i stay FLUX.2-dev. Gen pinned at the benched 512^2.

Rejected

  • Retro Diffusion (re-check) — flagship still API-only; Gumroad model SD1.5-era + non-commercial model license. Verdict stands.
  • Z-Image-Turbo as default — best fine detail in the field (only model to render crisp correct card indices) but typography spam under plain prompts + style drift. Weights + LoRA stay on RAID.

Deferred (with triggers, in docs/asset-gen-research.md 05.7)

Z-Image fine-detail lane · PixelOE contrast downscale in prod (torch dep) · SDXL family · 32-grid characters (every variant collapses to speckle — real open gap) · animation: wan2.2 attack LoRA pair is the non-locomotion candidate; object motion (card-flip) has nothing purpose-built — keyframes + RIFE is the technique to trial.

Fixture + critic fallout (same-day arc, full evidence in critic-calibration.md §6)

  • New critic correctly failed the old painterly knight → regenerated through the adopted path end-to-end (first production run of klein routing + trigger + palette post) → passes. Briefs ground-truthed.
  • Speed trial with the brain workstream: greedy temp 0 + per-verdict ms/token instrumentation kept; enable_thinking off was 6x faster but hallucinated ("static sheet" disproven by pixel diff) → reverted with a 4096 cap. Speed lever moves to the brain-server flags.
  • Fixture baseline: 6 pass / 1 fail (orange sentinel), deterministic across consecutive fresh rolls.

Calibration readout

Critic tracks the post-pass A/B on the incumbent (core FAIL -> oe PASS on the same raw, the leaf case) but does not crisply rank models — verdict-as-gate yes, verdict-as-ranker no; ranking stays with the eye.

Commits: eb94e83, d97e50e, 4344fb1, e2fba4c, dea5443, 81ceb98 + docs. My adopt/reject calls are provisional on your eye pass of the review page — pick_workflow is a one-line revert if the klein look isn't what you want.

## 05.7 complete — research, bench, decisions wired **Review page (eye test is yours):** http://trogdor:3923/games/sprite-bench/assets/review.html — 3 models x 2 prompt styles x 2 post-passes x 5 subjects x 2 grids, fixed seeds, critic verdicts alongside. Bench kit: trog-games/sprite-bench. ### Root cause of the #17 complaints The "mangled apple leaf" was **the post-pass, not the model**: in-graph `ImageQuantize` (median cut) merged small-area hues away — the model painted the leaf green every time. The "painterly mush" was prompt-level: plain briefs render airbrushed cartoons the post-pass then has to rescue. ### Adopted (deployed) - **Craft rubric** — `docs/pixel-art-craft.md` (saint11/Lospec vocabulary: clusters, hue-shifted ramps, sel-out, banding, pillow shading, grid budget) wired into the spec template and the visual critic. Critic enforcement is a style-block **Craft:** opt-in — legacy specs keep their honest bar, new games opt in via template. - **Craft prompting** — transforms the incumbent (deliberate clusters/sel-out *before* any post-pass), fixes Z-Image's typography spam. Caveat: sprite-phrased craft language hurts tiles; briefs stay kind-aware. - **CPU palette post** — `trog_lib/pixel_post.py`: OKLab k-means snapped to true source colors + orphan-pixel cleanup, replacing ImageQuantize in both flux2 graphs; applied to every PNG artifact, sheets included. - **klein-4B + Limbicnation pixel LoRA as the sprite/ui seat** — best cluster quality of the field at ~10-15s vs ~240s, Apache-2.0 base+LoRA. Klein pairs with the **Qwen3-4B TE** (Mistral dim-mismatches, live-caught). Backgrounds/i2i stay FLUX.2-dev. Gen pinned at the benched 512^2. ### Rejected - **Retro Diffusion (re-check)** — flagship still API-only; Gumroad model SD1.5-era + non-commercial model license. Verdict stands. - **Z-Image-Turbo as default** — best fine detail in the field (only model to render crisp correct card indices) but typography spam under plain prompts + style drift. Weights + LoRA stay on RAID. ### Deferred (with triggers, in docs/asset-gen-research.md 05.7) Z-Image fine-detail lane · PixelOE contrast downscale in prod (torch dep) · SDXL family · **32-grid characters (every variant collapses to speckle — real open gap)** · animation: wan2.2 attack LoRA pair is the non-locomotion candidate; object motion (card-flip) has nothing purpose-built — keyframes + RIFE is the technique to trial. ### Fixture + critic fallout (same-day arc, full evidence in critic-calibration.md §6) - New critic correctly failed the old painterly knight → **regenerated through the adopted path end-to-end** (first production run of klein routing + trigger + palette post) → passes. Briefs ground-truthed. - Speed trial with the brain workstream: greedy temp 0 + per-verdict ms/token instrumentation **kept**; `enable_thinking off` was 6x faster but **hallucinated** ("static sheet" disproven by pixel diff) → reverted with a 4096 cap. Speed lever moves to the brain-server flags. - **Fixture baseline: 6 pass / 1 fail (orange sentinel), deterministic across consecutive fresh rolls.** ### Calibration readout Critic tracks the post-pass A/B on the incumbent (core FAIL -> oe PASS on the same raw, the leaf case) but does not crisply rank models — verdict-as-gate yes, verdict-as-ranker no; ranking stays with the eye. Commits: eb94e83, d97e50e, 4344fb1, e2fba4c, dea5443, 81ceb98 + docs. My adopt/reject calls are provisional on your eye pass of the review page — pick_workflow is a one-line revert if the klein look isn't what you want.
cmoriarty commented 2026-07-25 17:00:31 -04:00
Author
Owner
Copy link

Eye-pick verdicts applied (final wiring, commit 29cfc22)

Picks from pick.html, all deployed and smoke-tested end-to-end:

  • Default image seat: FLUX.2-dev + craft prompting (klein lost the look vote 3-0; its graph stays available for a future speed-vs-look call — one-line route change buys 15s sprites). Craft prompting is now mechanical: style-block Craft: appends kind-aware suffixes, one-offs carry it by default.
  • PixelOE contrast-aware downscale in production (both post slides): graphs output raw renders, pixel_post.pixelize_png owns downscale + OKLab palette + orphan cleanup. torch-cpu/kornia/opencv joined the server image.
  • Z-Image fine-detail lane wired: Fine: field on spec entries + trog image --fine → zimage-pixel-t2i.json. For glyphs/indices/intricate small-grid subjects.
  • 32px guidance: complex → Fine lane, simple → default seat; no auto-routing (your knight/apple picks split).

Smoke proof (production one-offs, post-deploy): default lane apple — 96×96, exactly 16 colors, green leaf intact (the #17 failure, now structurally fixed); fine lane ace of spades — correct pip + indices.
http://trogdor:3923/oneoffs/2026-07-25/smoke-default-205815.png · http://trogdor:3923/oneoffs/2026-07-25/smoke-fine-205951.png

89 tests green. Ticket complete from my side — close when satisfied.

## Eye-pick verdicts applied (final wiring, commit 29cfc22) Picks from pick.html, all deployed and smoke-tested end-to-end: - **Default image seat: FLUX.2-dev + craft prompting** (klein lost the look vote 3-0; its graph stays available for a future speed-vs-look call — one-line route change buys 15s sprites). Craft prompting is now mechanical: style-block `Craft:` appends kind-aware suffixes, one-offs carry it by default. - **PixelOE contrast-aware downscale in production** (both post slides): graphs output raw renders, `pixel_post.pixelize_png` owns downscale + OKLab palette + orphan cleanup. torch-cpu/kornia/opencv joined the server image. - **Z-Image fine-detail lane wired**: `Fine:` field on spec entries + `trog image --fine` → `zimage-pixel-t2i.json`. For glyphs/indices/intricate small-grid subjects. - **32px guidance**: complex → Fine lane, simple → default seat; no auto-routing (your knight/apple picks split). Smoke proof (production one-offs, post-deploy): default lane apple — 96×96, exactly 16 colors, green leaf intact (the #17 failure, now structurally fixed); fine lane ace of spades — correct pip + indices. http://trogdor:3923/oneoffs/2026-07-25/smoke-default-205815.png · http://trogdor:3923/oneoffs/2026-07-25/smoke-fine-205951.png 89 tests green. Ticket complete from my side — close when satisfied.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
feat/11-walking-skeleton
No results found.
Labels
Clear labels   
bug
Something is not working

  
duplicate
This issue or pull request already exists

  
enhancement
New feature

  
help wanted
Need some help

  
invalid
Something is wrong

  
question
More information is needed

  
wontfix
This won't be fixed

No labels
bug
duplicate
enhancement
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
cmoriarty/trog#20
No description provided.