sprite & pixel-art quality bench — issue #20
  • HTML 78.5%
  • Python 19.7%
  • Shell 1.8%
Find a file
2026-07-25 16:39:31 -04:00
assets decision slideshow: pick.html — one open question per slide, prompts visible (#20) 2026-07-25 16:39:31 -04:00
bench decision slideshow: pick.html — one open question per slide, prompts visible (#20) 2026-07-25 16:39:31 -04:00
out bench results: raw metas, post variants, verdicts, review page (#20) 2026-07-25 15:53:54 -04:00
test-assets bench results: raw metas, post variants, verdicts, review page (#20) 2026-07-25 15:53:54 -04:00
test-raw bench results: raw metas, post variants, verdicts, review page (#20) 2026-07-25 15:53:54 -04:00
README.md bench results: raw metas, post variants, verdicts, review page (#20) 2026-07-25 15:53:54 -04:00

sprite-bench — trog ticket 05.7 (#20)

One-shot bench of sprite/pixel-art quality candidates against the incumbent image seat, 2026-07-25, on trogdor (4× Quadro RTX 5000 16GB, Turing sm_75). Results: assets/review.html (mirrored at http://trogdor:3923/games/sprite-bench/assets/review.html). Decisions live in trog's docs/asset-gen-research.md 05.7; the craft rubric in docs/pixel-art-craft.md.

Verdicts (short form)

  • Craft prompting — ADOPT. Transforms the incumbent (airbrushed cartoon → deliberate clusters/sel-out before any post-pass); fixes Z-Image's typography spam. Sprite-phrased craft language hurts tiles — keep briefs kind-aware.
  • PixelOE/OKLab post — ADOPT (palette part). Root-caused #17's mangled leaf: in-graph median-cut quantize merged small-area hues, not the model. Production got trog_lib.pixel_post (OKLab k-means + orphan cleanup); PixelOE's contrast downscale deferred (torch dep).
  • klein-4B + pixel LoRA — ADOPT for sprite/ui. Best clusters, ~10-15s vs ~240s, Apache-2.0 throughout. Weak on fine glyphs (card indices). Backgrounds/i2i stay FLUX.2-dev. Klein pairs with the Qwen3-4B TE — the Mistral TE dim-mismatches (live-caught).
  • Z-Image-Turbo — REJECT as default, DEFER as fine-detail lane. Only model to render crisp correct card indices; but typography spam under plain prompts + style drift. Weights stay on the RAID.
  • 32-grid characters — open gap. Every variant collapses to speckle on complex subjects at 32×32; simple subjects fine. Trigger + plan in the research doc.
  • Fixture knight regenerated through the adopted path end-to-end and passes the (greedy, thinking) critic; fixture baseline 6/1 restored.

The matrix

3 models × 2 prompt styles × 2 post-passes × 5 subjects × 2 grids, fixed seed per subject, same briefs everywhere:

  • models — FLUX.2-dev Q6 (incumbent) · FLUX.2-klein-4B Q6 + Limbicnation pixel LoRA (Apache-2.0) · Z-Image-Turbo Q8 + elusarca pixel LoRA (Apache-2.0)
  • prompt stylesplain (production behavior: brief + style suffix) · craft (adds the pixel-art craft vocabulary from docs/pixel-art-craft.md)
  • post-passescore (production: nearest downscale + median-cut quantize) · oe (PixelOE contrast-aware downscale + OKLab k-means palette + orphan-pixel cleanup) — applied laptop-side to the SAME raw image, so model and post are judged independently
  • subjects — knight (silhouette), card (fine detail), apple (organic), brick (tile), button (UI) at 96×96 and 32×32

Recipe (bench/ scripts)

  1. run-window.sh — ON trogdor: brain down, one ComfyUI seat (all four GPUs, port 18188), gen.py walks the matrix saving RAW full-res PNGs + meta to out/, brain restored on exit. gen.py --validate-only checks every graph node against the live /object_info first (use the --cpu container for that — no GPU window needed).
  2. post.py — laptop venv (pillow, numpy, opencv, pixeloe): raw → the two post variants at both grids into assets/.
  3. critique.py — real critics via the orchestrator's /critique (brain back up), 96-grid finals, verdicts into bench/verdicts.json.
  4. build-page.pyassets/review.html: pixel-true upscale + native speck, metadata chips, critic verdicts alongside for calibration.

Model/LoRA notes: all weights on the RAID (/mnt/models/comfyui/models). Z-Image's Qwen3-4B TE loads via CLIPLoader type=lumina2 — this ComfyUI build routes a QWEN3_4B text encoder to the Z-Image TE for every type except flux/flux2 (comfy/sd.py, live-verified; there is no z_image type in the enum).