Pixel art by placement: build the editor-MCP arm, and go learn the craft #38

Open
opened 2026-07-28 16:33:38 -04:00 by cmoriarty · 6 comments
Owner

Why now

#37's bake-off settled the direction. Same fixture, same brief, same grid,
same developer agent — only the generator differed:

asset diffusion + pixel_post brain writes a drawing program
robot 64x64 3.1% coverage, 3 colours 47.5%, 11 colours
battery 32x32 19.1%, 8 colours 81.6%, 5 colours
heart 32x32 27.4%, 5 colours 39.5%, 4 colours

Arm C's robot is a robot — symmetrical chassis, cyan eye panels, antennae,
shoulder pads, panelled torso, one consistent palette, clean outline,
filling its frame. The incumbent's is a fragment blob. Play them:
http://trogdor:3923/games/bakeoff/index.html

Arm C was deliberately the cheap proxy for the editor-MCP hypothesis —
deliberate pixel placement beats sampling-then-downscaling — built with a
6-op renderer and no dependencies precisely so the real investment would
only be made if the hypothesis held. It held. So build the real thing.

The other half of this ticket matters as much: arm C wins on placement
and still loses on craft. Its battery reads as a canister because it
dropped the yellow terminal cap the brief asked for. It has no notion of
ramps, dithering, sub-pixel weight, or how a real artist decides what to
cut at 32px. That knowledge exists; we have never gone and got it.

Part 1 — Arm B: a real pixel editor as an authoring seat

Candidates (already surveyed in docs/toolbelt-audit.md, scoped there as
post-processing; this reframes them as authoring):

Licensing decides the shape: Aseprite's binary is paid and its source
non-free (self-compile, personal use). LibreSprite
is the GPL fork and scripts headlessly in Lua or JS. Either way it is
CPU-only, deterministic, and does not contend with the brain for GPUs —
which is why it fits the seat model.

  • Pick the editor on licence + headless scriptability, not features
  • Container it (the vendoring pattern opencode/ and playtest/ use)
  • Expose it as an arm: ARMS["editor"] in trog_lib/bakeoff.py is one
    dict entry — the harness already holds everything else constant
  • By the MCP boundary rule (#13) it knows nothing about trog's domain,
    so if adopted it belongs in common-mcps with its own stack, the
    way searxng does — not in trog compose
  • Run it against the same fixture and compare with arm C, by eye

What would make it beat arm C: real dithering and ramp tools, layers,
sheet export, and animation frames — the things a 6-op renderer cannot do.
If it does not beat arm C, say so and keep the cheaper thing.

Part 2 — go and learn the craft, then bake it in

docs/pixel-art-craft.md exists (#20, from saint11/Lospec) and is wired
into the critic as a style-block opt-in. It is a rubric for JUDGING. What we
do not have is craft knowledge shaping how art gets AUTHORED — which is now
the binding constraint, because arm C proves we can place any pixel we like
and the open question is which pixels.

Research targets:

  • How pixel artists actually work at small sizes. Silhouette first,
    readability at the played size, what gets cut at 32px vs 64px, why
    the eye reads a shape in a quarter second. saint11, Pedro Medeiros,
    Lospec tutorials, MortMort, Adam Saltsman's writing.
  • Palette discipline. Ramps and hue shifting (shadows toward blue,
    highlights toward warm), why a 16-colour game palette beats
    per-asset palettes, how Celeste/Dead Cells/Stardew constrain
    themselves. We now enforce one palette per game mechanically
    (8ba4c3b) — the open question is how to CHOOSE it well.
  • The games named as the bar. Celeste (tiny palette, hand-placed,
    readable at speed), Stardew (warm consistent light, dense but legible
    tiles), Terraria (huge asset count, one coherent look), Dead Cells
    (3D-rendered then hand-retouched into pixel art — a genuinely
    different pipeline worth understanding, and possibly a third arm).
  • Dead Cells' pipeline specifically. They model/animate in 3D and
    convert to sprites. If that is tractable it sidesteps both the
    diffusion and the placement problem for ANIMATION, which is our
    weakest area.
  • Animation craft. Frame counts, key poses, smear frames,
    anticipation. pix3lwalk covers walk cycles only; #20 found nothing
    purpose-built for object motion.

Output, and this is the part that must not be skipped: every finding
lands as something mechanical, not as advice in a document.
#20's
research was correct and evaporated because its conclusion was addressed to
a human who is not in an unattended run. So each lesson becomes one of:
a standing prompt suffix, a pixelprog op or system-prompt rule, a
deterministic gate in critics.cutout_problem, or a floor in assets.py.

If a finding cannot be made mechanical, it does not count as done — write
down explicitly who is supposed to act on it and when.

Part 3 — settle why the incumbent was so bad

Not to rescue it, but because we do not understand our own tools and that
is its own risk. Leading suspect: the cutout is eating dark subjects.
Arm A's robot came back 3 colours at 3.1% coverage on a dark navy dusk
palette; cutout_background flood-fills from the border through
same-coloured pixels, and a slate robot against a dark backdrop is exactly
where subject and background quantise close enough to merge. The bright red
heart survived; the dark robot did not.

  • Re-run the same brief with the cutout disabled and compare coverage
  • If confirmed: refuse a cutout that removes more than N% of the frame,
    and/or force a contrasting backdrop in the prompt
  • Eliminate the other suspects in the same pass: gen-at-512 downscaled
    16x to a 32 grid; PixelOE contrast downscale on dark low-contrast
    art; the craft suffix's "centered on a plain background" fighting
    "fill the frame"
  • gen-at-256#20's own revisit trigger fired (recorded in
    docs/asset-gen-research.md) and the revisit has still not happened

Bar

Same as always: open weights or self-hostable, commercially usable licence,
no phone-home. Hosted pixel-art SaaS (PixelLab,
Retro Diffusion) stays excluded for the same
reason the Phaser Game Agent MCP did — worth ONE manual run as a reference
ceiling, so we know what we are giving up.

Judged the way #37 judged: by playing the game, not by looking at a sprite
sheet. The harness is trog test bakeoff.

## Why now #37's bake-off settled the direction. Same fixture, same brief, same grid, same developer agent — only the generator differed: | asset | diffusion + pixel_post | brain writes a drawing program | |---|---|---| | robot 64x64 | 3.1% coverage, 3 colours | **47.5%, 11 colours** | | battery 32x32 | 19.1%, 8 colours | **81.6%, 5 colours** | | heart 32x32 | 27.4%, 5 colours | **39.5%, 4 colours** | Arm C's robot is a robot — symmetrical chassis, cyan eye panels, antennae, shoulder pads, panelled torso, one consistent palette, clean outline, filling its frame. The incumbent's is a fragment blob. Play them: http://trogdor:3923/games/bakeoff/index.html Arm C was deliberately the *cheap proxy* for the editor-MCP hypothesis — deliberate pixel placement beats sampling-then-downscaling — built with a 6-op renderer and no dependencies precisely so the real investment would only be made if the hypothesis held. **It held.** So build the real thing. The other half of this ticket matters as much: arm C wins on *placement* and still loses on *craft*. Its battery reads as a canister because it dropped the yellow terminal cap the brief asked for. It has no notion of ramps, dithering, sub-pixel weight, or how a real artist decides what to cut at 32px. That knowledge exists; we have never gone and got it. ## Part 1 — Arm B: a real pixel editor as an authoring seat Candidates (already surveyed in `docs/toolbelt-audit.md`, scoped there as *post-processing*; this reframes them as **authoring**): - [willibrandon/pixel-mcp](https://github.com/willibrandon/pixel-mcp) — animation, retro palettes, dithering, shading, spritesheet export - [rkdfx/aseprite-mcp](https://github.com/rkdfx/aseprite-mcp) — Lua scripting - [diivi/aseprite-mcp](https://enterprisedna.co/directories/mcp/diivi-aseprite-mcp/) Licensing decides the shape: Aseprite's binary is paid and its source non-free (self-compile, personal use). [LibreSprite](https://github.com/LibreSprite/LibreSprite) is the GPL fork and scripts headlessly in Lua or JS. Either way it is CPU-only, deterministic, and does not contend with the brain for GPUs — which is why it fits the seat model. - [ ] Pick the editor on licence + headless scriptability, not features - [ ] Container it (the vendoring pattern `opencode/` and `playtest/` use) - [ ] Expose it as an arm: `ARMS["editor"]` in `trog_lib/bakeoff.py` is one dict entry — the harness already holds everything else constant - [ ] By the MCP boundary rule (#13) it knows nothing about trog's domain, so if adopted it belongs in **common-mcps** with its own stack, the way searxng does — not in trog compose - [ ] Run it against the same fixture and compare with arm C, by eye **What would make it beat arm C:** real dithering and ramp tools, layers, sheet export, and animation frames — the things a 6-op renderer cannot do. If it does not beat arm C, say so and keep the cheaper thing. ## Part 2 — go and learn the craft, then bake it in `docs/pixel-art-craft.md` exists (#20, from saint11/Lospec) and is wired into the critic as a style-block opt-in. It is a rubric for JUDGING. What we do not have is craft knowledge shaping how art gets AUTHORED — which is now the binding constraint, because arm C proves we can place any pixel we like and the open question is which pixels. Research targets: - [ ] **How pixel artists actually work at small sizes.** Silhouette first, readability at the played size, what gets cut at 32px vs 64px, why the eye reads a shape in a quarter second. saint11, Pedro Medeiros, Lospec tutorials, MortMort, Adam Saltsman's writing. - [ ] **Palette discipline.** Ramps and hue shifting (shadows toward blue, highlights toward warm), why a 16-colour game palette beats per-asset palettes, how Celeste/Dead Cells/Stardew constrain themselves. We now enforce one palette per game mechanically (`8ba4c3b`) — the open question is how to CHOOSE it well. - [ ] **The games named as the bar.** Celeste (tiny palette, hand-placed, readable at speed), Stardew (warm consistent light, dense but legible tiles), Terraria (huge asset count, one coherent look), Dead Cells (3D-rendered then hand-retouched into pixel art — a genuinely different pipeline worth understanding, and possibly a third arm). - [ ] **Dead Cells' pipeline specifically.** They model/animate in 3D and convert to sprites. If that is tractable it sidesteps both the diffusion and the placement problem for ANIMATION, which is our weakest area. - [ ] **Animation craft.** Frame counts, key poses, smear frames, anticipation. pix3lwalk covers walk cycles only; #20 found nothing purpose-built for object motion. Output, and this is the part that must not be skipped: **every finding lands as something mechanical, not as advice in a document.** #20's research was correct and evaporated because its conclusion was addressed to a human who is not in an unattended run. So each lesson becomes one of: a standing prompt suffix, a `pixelprog` op or system-prompt rule, a deterministic gate in `critics.cutout_problem`, or a floor in `assets.py`. If a finding cannot be made mechanical, it does not count as done — write down explicitly who is supposed to act on it and when. ## Part 3 — settle why the incumbent was so bad Not to rescue it, but because we do not understand our own tools and that is its own risk. Leading suspect: **the cutout is eating dark subjects.** Arm A's robot came back 3 colours at 3.1% coverage on a dark navy dusk palette; `cutout_background` flood-fills from the border through same-coloured pixels, and a slate robot against a dark backdrop is exactly where subject and background quantise close enough to merge. The bright red heart survived; the dark robot did not. - [ ] Re-run the same brief with the cutout disabled and compare coverage - [ ] If confirmed: refuse a cutout that removes more than N% of the frame, and/or force a contrasting backdrop in the prompt - [ ] Eliminate the other suspects in the same pass: gen-at-512 downscaled 16x to a 32 grid; PixelOE contrast downscale on dark low-contrast art; the craft suffix's "centered on a plain background" fighting "fill the frame" - [ ] **gen-at-256** — #20's own revisit trigger fired (recorded in `docs/asset-gen-research.md`) and the revisit has still not happened ## Bar Same as always: open weights or self-hostable, commercially usable licence, no phone-home. Hosted pixel-art SaaS ([PixelLab](https://www.pixellab.ai/), [Retro Diffusion](https://retrodiffusion.ai/)) stays excluded for the same reason the Phaser Game Agent MCP did — worth ONE manual run as a reference ceiling, so we know what we are giving up. Judged the way #37 judged: by playing the game, not by looking at a sprite sheet. The harness is `trog test bakeoff`.
Author
Owner

Part 1 decided and built; first editor-arm bake-off run is in

Session of 2026-07-28 (evening). Commits d3c159f, a7d23a8 — pushed, CI green, deployed.

The licence survey settled the editor pick, and the pick is: no binary

  • Aseprite family (willibrandon/pixel-mcp, rkdfx, diivi): dead on licence. Every one drives the Aseprite binary; EULA §2(g) permits compiling "for your own personal purpose or to propose a contribution" only. A commercial pipeline seat fails the bar however MIT the wrapper is — and pixel-mcp hard-requires aseprite_path for all functionality (no LibreSprite mode).
  • LibreSprite: passes licence (GPL), fails capability. Headless batch works, but the scripting API is getPixel/putPixel, layers, palettes — a strict subset of what pixelprog already does, with none of the dithering/shading/ramp tools that were the whole argument for an editor (those are post-fork Aseprite features). Its MCP wrappers attach to a running GUI over a port; "hacky and brittle" is their own README's phrase.

So the real thing the winning hypothesis justified is the editor's toolset, not an editor binary: trog_lib/pixeledit.py, arm editor in the bake-off. Flood fill, deliberate dither (3 named patterns), directional auto-shade, selective outline — plus part 2's craft rules enforced, not advised (the #20 lesson, applied):

  • a ramp that never shifts hue is rejected at parse, with the fix in the error the retry loop hands back
  • dither on grids ≤32px is refused — at that size it is noise by rule 10
  • one declared light source; shade/selout derive every highlight from it, so pillow shading cannot come out of the op
  • silhouette-first staged workflow + per-grid detail budgets in the system prompt

docs/pixel-art-craft.md now carries the rule→mechanism table. The findings that could not be made mechanical (game-palette choice, animation frames, the Dead Cells 3D→sprite pipeline as a possible third arm) are listed there with a named owner and trigger each — not as advice. docs/toolbelt-audit.md's aseprite row is now rejected with the reasoning. willibrandon/pixel-mcp (MIT) stays a reference for dither/shade algorithms.

First run: trog test bakeoff --arms commands,editor

commands (frozen control) editor (#38)
rooftop-dusk 480x270 100%, critic fail 100%, critic fail
robot 64 47%, critic fail 30%, critic pass
battery 32 82%, critic pass 48%, critic fail
heart 32 39%, critic fail 69%, critic fail
build ok ok
plays 59.3 fps 59.2 fps
check_phase verified FALSE — assets_used: committed but never loaded TRUE — every asset loaded and used

Play them (the verdict, per the house rule):

Read of the run: the editor arm went 4-for-4 through the new gates (zero unusable programs), and its craft mechanics all fired — the robot has a consistent top-left light, hue-shifted ramps on the limbs, and no dead uniform outline; it is the best sprite either program arm has produced. By eye the generator comparison is otherwise mixed (control's battery is more readable; both hearts are weak; both backgrounds are primitive gradient-bands — composition is a different problem from craft ops). The control losing on assets_used is build-session variance, not generator quality — worth remembering when reading the verified column.

Open, in order

  1. Value contrast is the next craft lever: shade + ramps made the editor's sprites read muddier than the control's bold flats at 1x. Candidate mechanism: a value-structure rule (big value separation between adjacent forms) in the prompt, possibly a deterministic palette value-spread check.
  2. Persist the program JSON beside the PNG — the editor heart has odd interior holes and there is no way to diagnose them because the program is thrown away after render.
  3. Part 3 autopsy (cutout eating dark subjects) — raw renders are not persisted, so it needs a regeneration pass; gen-at-256 still owed with it.
  4. Backgrounds want composition vocabulary (skyline silhouette, cloud shapes), not more pixel ops.
## Part 1 decided and built; first editor-arm bake-off run is in Session of 2026-07-28 (evening). Commits `d3c159f`, `a7d23a8` — pushed, CI green, deployed. ### The licence survey settled the editor pick, and the pick is: no binary - **Aseprite family (willibrandon/pixel-mcp, rkdfx, diivi): dead on licence.** Every one drives the Aseprite binary; EULA §2(g) permits compiling *"for your own personal purpose or to propose a contribution"* only. A commercial pipeline seat fails the bar however MIT the wrapper is — and pixel-mcp hard-requires `aseprite_path` for all functionality (no LibreSprite mode). - **LibreSprite: passes licence (GPL), fails capability.** Headless batch works, but the scripting API is getPixel/putPixel, layers, palettes — a **strict subset of what `pixelprog` already does**, with none of the dithering/shading/ramp tools that were the whole argument for an editor (those are post-fork Aseprite features). Its MCP wrappers attach to a *running GUI* over a port; "hacky and brittle" is their own README's phrase. So the real thing the winning hypothesis justified is the editor's **toolset**, not an editor binary: `trog_lib/pixeledit.py`, arm `editor` in the bake-off. Flood fill, deliberate dither (3 named patterns), directional auto-shade, selective outline — plus part 2's craft rules **enforced, not advised** (the #20 lesson, applied): - a ramp that never shifts hue is rejected at parse, with the fix in the error the retry loop hands back - dither on grids ≤32px is refused — at that size it is noise by rule 10 - one declared light source; `shade`/`selout` derive every highlight from it, so pillow shading cannot come out of the op - silhouette-first staged workflow + per-grid detail budgets in the system prompt `docs/pixel-art-craft.md` now carries the rule→mechanism table. The findings that could **not** be made mechanical (game-palette *choice*, animation frames, the Dead Cells 3D→sprite pipeline as a possible third arm) are listed there with a named owner and trigger each — not as advice. `docs/toolbelt-audit.md`'s aseprite row is now **rejected** with the reasoning. willibrandon/pixel-mcp (MIT) stays a reference for dither/shade algorithms. ### First run: `trog test bakeoff --arms commands,editor` | | commands (frozen control) | editor (#38) | |---|---|---| | rooftop-dusk 480x270 | 100%, critic fail | 100%, critic fail | | robot 64 | 47%, critic fail | 30%, **critic pass** | | battery 32 | 82%, critic pass | 48%, critic fail | | heart 32 | 39%, critic fail | 69%, critic fail | | build | ok | ok | | plays | 59.3 fps | 59.2 fps | | **check_phase verified** | **FALSE — `assets_used`: committed but never loaded** | **TRUE — every asset loaded and used** | Play them (the verdict, per the house rule): - compare: http://trogdor:3923/games/bakeoff/index.html - editor: http://trogdor:3923/games/bakeoff-editor/index.html — **a real game.** "ROOFTOP SALVAGE": dusk background composited, the robot walks the rooftop, batteries fall, the hearts are the lives UI, score works. - commands: http://trogdor:3923/games/bakeoff-commands/index.html — this run's build drew green placeholder rectangles over its own committed art. The one failure the slice brief forbids, and `assets_used` caught it deterministically. **Read of the run:** the editor arm went 4-for-4 through the new gates (zero unusable programs), and its craft mechanics all fired — the robot has a consistent top-left light, hue-shifted ramps on the limbs, and no dead uniform outline; it is the best sprite either program arm has produced. By eye the *generator* comparison is otherwise mixed (control's battery is more readable; both hearts are weak; both backgrounds are primitive gradient-bands — composition is a different problem from craft ops). The control losing on `assets_used` is build-session variance, not generator quality — worth remembering when reading the verified column. ### Open, in order 1. **Value contrast** is the next craft lever: `shade` + ramps made the editor's sprites read muddier than the control's bold flats at 1x. Candidate mechanism: a value-structure rule (big value separation between adjacent forms) in the prompt, possibly a deterministic palette value-spread check. 2. **Persist the program JSON beside the PNG** — the editor heart has odd interior holes and there is no way to diagnose them because the program is thrown away after render. 3. **Part 3 autopsy** (cutout eating dark subjects) — raw renders are not persisted, so it needs a regeneration pass; gen-at-256 still owed with it. 4. Backgrounds want composition vocabulary (skyline silhouette, cloud shapes), not more pixel ops.
Author
Owner

Brain bake-off: four hands, one pencil — and the biggest hand did not win

Session of 2026-07-28 (night). Same generator (pixeledit), same fixture, same developer (BUILD_MODEL = qwen3.6-35b-a3b) — only the model writing the drawing programs varied. Commits c0b67ff (harness: --models matrix, per-model repos, programs committed beside PNGs, per-asset gallery), 74a7516 (gate fix, below). All deployed, CI green, stack env restored to trogdor-speed.

Compare + play all four (per-asset gallery with coverage/ops/ramps/draw-time under each sprite):
http://trogdor:3923/games/bakeoff/index.html

35B-A3B Q4 (baseline) 27B Q8 Qwen3.5-122B-A10B IQ3_S Gemma 4 31B QAT
assets drawn 4/4 3/4 2/4 2/4
robot 47%, 19 ops 57%, 21 ops 69%, 14 ops 52%, 17 ops
heart 44% 93% (6-op blob) 18% 24%
draw time, 4 assets 19 min 104 min 30 min 41 min
gen speed fastest ~3x slower than 122B 32.6 tok/s mid
plays / verified plays, dev ignored art plays + verified stalled (infra, see below) plays + verified

The eye's read

  • 27B-Q8 drew the best sprite of the whole experiment: chunky robot, orange eyes, warm rim light on the lit side, real value structure — and the only battery with terminal studs. Its heart is an over-filled blob. It is also brutally slow: 104 minutes of drawing.
  • 122B is bold but off-brief: it filled 69% of the frame with a giant robot head — the brief asks for a standing robot with catching arms. Scale did not buy brief fidelity.
  • Gemma's robot is the cleanest silhouette (proper sel-out, tidy) but simplest; it loses most of its assets to one habit (below).
  • The baseline remains the value pick: only model to land 4/4, competent everywhere, 2-5x faster than everything else.

What actually separated the models: the craft gates

The strict dialect is now the dominant failure mode, and it discriminates by model habit:

  • Gemma writes flat-hue ramps. 3 of its 4 losses across both attempts were ramp never shifts hue, and it did not correct from the error text within the 2-try budget. 122B lost two the same way; the baseline (which grew up on this dialect) lost none.
  • 27B-Q8 writes 8-digit hex (#00000000 — the RGBA idiom for transparent) and the parser only takes 6.
  • Five assets died at these gates in one run. The gates are teaching the right craft and killing real art while doing it.

Round-3 dialect changes (apply to ALL models equally, never mid-experiment): accept 8-digit hex and fold alpha into the transparency contract; put a concrete example ramp in the hue-shift error; consider 3 tries; possibly auto-repair a flat ramp (mechanically bend shadows cool / highlights warm) instead of rejecting.

Infra incident, root-caused and fixed

The 122B build's developer called generate_asset mid-session. The /jobs gate keyed on _build_task — but a bake-off arm's build runs inside _bakeoff_task — so the request passed, the image seat took the brain's cards under the non-resident brains profile, and the agent's own backend died under it. Build stalled 600s; the next arm's draws failed on connection errors until the seat TTL restored the brain. Fixed in 74a7516: codeseat.STATE.status == "running" is the fact the gate reads now, whatever task wraps the session. Gemma got a clean re-run after the fix (plays + verified); the page carries the merged results.

The other standing result

assets_used failed on 2 of 4 builds — the shared developer wires committed art into the game roughly half the time, independent of who drew it. That is now the biggest non-art quality gate. Candidate: one repair session keyed on a failed assets_used check, inside the existing BUILD_FIX_RETRIES budget.

Bigger models, round 2 (queued, not run)

DeepSeek-V4-Flash and Hy3 were downloaded and then dropped (no vision — user call); weights deleted. Qwen3.5-397B-A17B UD-Q3_K_XL (179G, vision-capable) is on the RAID awaiting a roster entry with CPU expert offload (-ncmoe) — it cannot fit 64GB VRAM.

Playable demos

## Brain bake-off: four hands, one pencil — and the biggest hand did not win Session of 2026-07-28 (night). Same generator (`pixeledit`), same fixture, same developer (BUILD_MODEL = qwen3.6-35b-a3b) — only the model writing the drawing programs varied. Commits `c0b67ff` (harness: `--models` matrix, per-model repos, programs committed beside PNGs, per-asset gallery), `74a7516` (gate fix, below). All deployed, CI green, stack env restored to trogdor-speed. **Compare + play all four** (per-asset gallery with coverage/ops/ramps/draw-time under each sprite): http://trogdor:3923/games/bakeoff/index.html | | 35B-A3B Q4 (baseline) | 27B Q8 | Qwen3.5-122B-A10B IQ3_S | Gemma 4 31B QAT | |---|---|---|---|---| | assets drawn | **4/4** | 3/4 | 2/4 | 2/4 | | robot | 47%, 19 ops | **57%, 21 ops** | 69%, 14 ops | 52%, 17 ops | | heart | 44% | 93% (6-op blob) | 18% | 24% | | draw time, 4 assets | **19 min** | 104 min | 30 min | 41 min | | gen speed | fastest | ~3x slower than 122B | 32.6 tok/s | mid | | plays / verified | plays, dev ignored art | **plays + verified** | stalled (infra, see below) | **plays + verified** | ### The eye's read - **27B-Q8 drew the best sprite of the whole experiment**: chunky robot, orange eyes, warm rim light on the lit side, real value structure — and the only battery with terminal studs. Its heart is an over-filled blob. It is also brutally slow: 104 minutes of drawing. - **122B is bold but off-brief**: it filled 69% of the frame with a giant robot *head* — the brief asks for a standing robot with catching arms. Scale did not buy brief fidelity. - **Gemma's robot is the cleanest silhouette** (proper sel-out, tidy) but simplest; it loses most of its assets to one habit (below). - **The baseline remains the value pick**: only model to land 4/4, competent everywhere, 2-5x faster than everything else. ### What actually separated the models: the craft gates The strict dialect is now the dominant failure mode, and it discriminates **by model habit**: - **Gemma writes flat-hue ramps.** 3 of its 4 losses across both attempts were `ramp never shifts hue`, and it did not correct from the error text within the 2-try budget. 122B lost two the same way; the baseline (which grew up on this dialect) lost none. - **27B-Q8 writes 8-digit hex** (`#00000000` — the RGBA idiom for transparent) and the parser only takes 6. - Five assets died at these gates in one run. The gates are teaching the right craft and killing real art while doing it. **Round-3 dialect changes** (apply to ALL models equally, never mid-experiment): accept 8-digit hex and fold alpha into the transparency contract; put a concrete example ramp in the hue-shift error; consider 3 tries; possibly auto-repair a flat ramp (mechanically bend shadows cool / highlights warm) instead of rejecting. ### Infra incident, root-caused and fixed The 122B build's developer called `generate_asset` mid-session. The /jobs gate keyed on `_build_task` — but a bake-off arm's build runs inside `_bakeoff_task` — so the request passed, the image seat took the brain's cards under the non-resident brains profile, and **the agent's own backend died under it**. Build stalled 600s; the next arm's draws failed on connection errors until the seat TTL restored the brain. Fixed in `74a7516`: `codeseat.STATE.status == "running"` is the fact the gate reads now, whatever task wraps the session. Gemma got a clean re-run after the fix (plays + verified); the page carries the merged results. ### The other standing result `assets_used` failed on 2 of 4 builds — the shared developer wires committed art into the game roughly half the time, independent of who drew it. That is now the biggest non-art quality gate. Candidate: one repair session keyed on a failed `assets_used` check, inside the existing `BUILD_FIX_RETRIES` budget. ### Bigger models, round 2 (queued, not run) DeepSeek-V4-Flash and Hy3 were downloaded and then dropped (no vision — user call); weights deleted. **Qwen3.5-397B-A17B UD-Q3_K_XL (179G, vision-capable) is on the RAID** awaiting a roster entry with CPU expert offload (`-ncmoe`) — it cannot fit 64GB VRAM. ### Playable demos - baseline: http://trogdor:3923/games/bakeoff-editor-qwen3-6-35b-a3b/index.html - 27B-Q8: http://trogdor:3923/games/bakeoff-editor-qwen3-6-27b-q8/index.html - 122B: http://trogdor:3923/games/bakeoff-editor-qwen3-5-122b-a10b/index.html - Gemma: http://trogdor:3923/games/bakeoff-editor-gemma-4-31b/index.html
Author
Owner

Round 3: the editor got eyes, and the background got a skyline

Session of 2026-07-29 (early). Commits 28bdbb1 (draw–look–revise, gate auto-repair, per-model coding, assets_used repair session), a6f787f (revision no-regression guard). Deployed, CI green, run under plain trogdor-speed with the resident 35B-A3B.

Play it: http://trogdor:3923/games/bakeoff-editor/index.html · gallery: http://trogdor:3923/games/bakeoff/index.html

What changed

  • draw–look–revise: after a good render the model is shown its own sprite at inspection scale and returns a full corrected program, up to PIXELEDIT_REVISIONS (3) looks. KEEP ends early.
  • Gates repair instead of reject where the fix is mechanical: flat-hue ramps get bent (shadows cool, highlights warm) with the repair disclosed; 8-digit hex accepted, alpha dropped. Rejection had zeroed five assets across three models.
  • The developer follows the arm's model (codeseat.build(model=)).
  • One assets_used repair session when a build ships art it never loads.

What the first unguarded run taught (and the guard it forced)

The model answers "revise" with NEAR-EMPTY programs despite being asked for complete ones: a 100% background fell to 2%, the battery to 30%, the heart to 11%. a6f787f: a revision that halves coverage is refused, costs its slot, and the refusal is worded into the next look. The guarded rerun immediately behaved.

The guarded run

asset coverage revisions critic
rooftop-dusk 100% 3 fail (harsh — see below)
robot 47% 1 pass
battery 80% 2 pass
heart 31% 3 fail

Build shipped dead art → the repair session fired for the first time and fixed it: plays, verified, every asset loaded. That closes the loop nothing else had closed — art reaching the screen is now enforced, not hoped for.

The background is the story. For the first time in any run of any arm it is a composition: banded dusk sky fading indigo→orange, cloud silhouettes, stars, a building skyline, deliberate dither texture on the warm band and the rooftop, one detail cluster (an AC unit) off-centre. That is the composition prompt + the dither op + three looks at its own render, compounding. Two critic passes on sprites is also the best bare-run showing. The heart finally has lobes, hue-shifted shading and a highlight — its coverage (31%) is honest for the shape.

The critic still fails the background; on the evidence of the in-play frame the critic is now behind the eye on backgrounds — calibration note for #18's successor, not a gate change tonight.

Standing

  • Revise loop measurably pays: every asset that took revisions kept or improved coverage under the guard, and the scene reads as one game.
  • gen-at-256 (part 3) deliberately still open — arm A is no longer the pixel path.
  • Next levers, in order: heart-class silhouettes at 32px (budget prompt refinement), critic calibration on backgrounds, palette-choice mechanism (owner/trigger unchanged in docs/pixel-art-craft.md), 397B-A17B (vision, on RAID at IQ3_S + IQ4_XS) when model comparison becomes interesting again.
## Round 3: the editor got eyes, and the background got a skyline Session of 2026-07-29 (early). Commits `28bdbb1` (draw–look–revise, gate auto-repair, per-model coding, assets_used repair session), `a6f787f` (revision no-regression guard). Deployed, CI green, run under plain trogdor-speed with the resident 35B-A3B. **Play it:** http://trogdor:3923/games/bakeoff-editor/index.html · gallery: http://trogdor:3923/games/bakeoff/index.html ### What changed - **draw–look–revise**: after a good render the model is shown its own sprite at inspection scale and returns a full corrected program, up to `PIXELEDIT_REVISIONS` (3) looks. KEEP ends early. - **Gates repair instead of reject** where the fix is mechanical: flat-hue ramps get bent (shadows cool, highlights warm) with the repair disclosed; 8-digit hex accepted, alpha dropped. Rejection had zeroed five assets across three models. - **The developer follows the arm's model** (`codeseat.build(model=)`). - **One `assets_used` repair session** when a build ships art it never loads. ### What the first unguarded run taught (and the guard it forced) The model answers "revise" with NEAR-EMPTY programs despite being asked for complete ones: a 100% background fell to 2%, the battery to 30%, the heart to 11%. `a6f787f`: a revision that halves coverage is refused, costs its slot, and the refusal is worded into the next look. The guarded rerun immediately behaved. ### The guarded run | asset | coverage | revisions | critic | |---|---|---|---| | rooftop-dusk | 100% | 3 | fail (harsh — see below) | | robot | 47% | 1 | **pass** | | battery | 80% | 2 | **pass** | | heart | 31% | 3 | fail | Build shipped dead art → **the repair session fired for the first time and fixed it**: plays, verified, every asset loaded. That closes the loop nothing else had closed — art reaching the screen is now enforced, not hoped for. **The background is the story.** For the first time in any run of any arm it is a *composition*: banded dusk sky fading indigo→orange, cloud silhouettes, stars, a building skyline, deliberate dither texture on the warm band and the rooftop, one detail cluster (an AC unit) off-centre. That is the composition prompt + the dither op + three looks at its own render, compounding. Two critic passes on sprites is also the best bare-run showing. The heart finally has lobes, hue-shifted shading and a highlight — its coverage (31%) is honest for the shape. The critic still fails the background; on the evidence of the in-play frame the critic is now behind the eye on backgrounds — calibration note for #18's successor, not a gate change tonight. ### Standing - Revise loop measurably pays: every asset that took revisions kept or improved coverage under the guard, and the scene reads as one game. - gen-at-256 (part 3) deliberately still open — arm A is no longer the pixel path. - Next levers, in order: heart-class silhouettes at 32px (budget prompt refinement), critic calibration on backgrounds, palette-choice mechanism (owner/trigger unchanged in docs/pixel-art-craft.md), 397B-A17B (vision, on RAID at IQ3_S + IQ4_XS) when model comparison becomes interesting again.
Author
Owner

The naked baseline: seven minutes, 733 lines, dead on arrival

Session of 2026-07-29. The null hypothesis the whole studio has to beat (commits 5237ab4, 85282f6, 1df8a53): one agent, the brief inline, Phaser 4 skills + rexUI + web search as materials, the Q8/262k trogdor-coder brain, no process — no design docs, no critics, no revise, no repair, no playtest tool, no time cap. Judged after the fact by the same recorded playtest as every arm.

Result

naked baseline round-3 editor game (process)
session 7 min, one shot, declared done draw+revise ~25 min, build, 1 repair pass
art code-drawn Graphics textures (its own answer to having no asset pipeline) pixel-art scene: composed dusk background, sprites, one palette
plays FALSE — black screen TRUE, verified, all assets wired
cause scene.make.graphics({add:false}) — a Phaser 3 idiom, line 28, first call of its texture pipeline → Cannot read properties of undefined (reading 'sys') → one uncaught error kills all 733 lines

The agent read the skills (native skill tool fired — the new discovery symlinks work), wrote a complete, confident, well-structured game around ONE v3-ism it never saw fail — because the one thing it lacked was any way to RUN the game. It did not use rexUI; it did not search the web; it built exactly what it believed and shipped it blind. AGENTS.md's black-screen lore, reproduced from scratch on the first try.

Reading

  • The minimum thing "the process" is worth is an eye. Every layer we removed, the decisive one was playtest_build (ours). A coding agent without execution feedback ships confident corpses; with it, the same model iterates to a playing game every time it has run under the harness.
  • The 733 lines are genuinely competent — structure, difficulty ramp, particles, screen-shake — which makes the point sharper: capability was never the gap. Feedback was.
  • One run, n=1, stated as such. A rerun would likely die on a different invented call — the failure class, not the specific line, is the finding.

Infra that landed with this (all deployed)

  • trogdor-coder profile/roster: 35B-A3B Q8 on all four cards at the full native 262,144 ctx (fits in 54 of 64 GiB; the 131k limit was a pair budget, not a model limit)
  • Attachable build sessions: OPENCODE_ATTACH_PORT=4096 publishes the seat; GET /build carries the per-session password — opencode attach http://trogdor:4096 and you watch the developer live
  • Phaser skills symlinked into OpenCode's native skill discovery, so sessions with no AGENTS.md still see them; searxng MCP wired into the seat (per-session); rexUI (MIT, official Phaser 4 port, pinned) named trusted in AGENTS.md
  • Correction for the record: an earlier claim here that the Phaser skills were never wired was false — a grep filtered lines containing "kill", which excludes "skill". They were vendored, pinned, and guarded all along.

Stack env restored to trogdor-speed. Play the corpse: http://trogdor:3923/games/bakeoff-baseline/index.html · the comparison page has its card.

## The naked baseline: seven minutes, 733 lines, dead on arrival Session of 2026-07-29. The null hypothesis the whole studio has to beat (commits `5237ab4`, `85282f6`, `1df8a53`): one agent, the brief inline, Phaser 4 skills + rexUI + web search as materials, the Q8/262k `trogdor-coder` brain, **no process** — no design docs, no critics, no revise, no repair, no playtest tool, no time cap. Judged after the fact by the same recorded playtest as every arm. ### Result | | naked baseline | round-3 editor game (process) | |---|---|---| | session | **7 min**, one shot, declared done | draw+revise ~25 min, build, 1 repair pass | | art | code-drawn `Graphics` textures (its own answer to having no asset pipeline) | pixel-art scene: composed dusk background, sprites, one palette | | plays | **FALSE — black screen** | TRUE, verified, all assets wired | | cause | `scene.make.graphics({add:false})` — a Phaser **3** idiom, line 28, first call of its texture pipeline → `Cannot read properties of undefined (reading 'sys')` → one uncaught error kills all 733 lines | — | The agent read the skills (native skill tool fired — the new discovery symlinks work), wrote a complete, confident, well-structured game around ONE v3-ism it never saw fail — because the one thing it lacked was any way to RUN the game. It did not use rexUI; it did not search the web; it built exactly what it believed and shipped it blind. AGENTS.md's black-screen lore, reproduced from scratch on the first try. ### Reading - **The minimum thing "the process" is worth is an eye.** Every layer we removed, the decisive one was `playtest_build` (ours). A coding agent without execution feedback ships confident corpses; with it, the same model iterates to a playing game every time it has run under the harness. - The 733 lines are genuinely competent — structure, difficulty ramp, particles, screen-shake — which makes the point sharper: capability was never the gap. Feedback was. - One run, n=1, stated as such. A rerun would likely die on a different invented call — the failure class, not the specific line, is the finding. ### Infra that landed with this (all deployed) - `trogdor-coder` profile/roster: 35B-A3B **Q8** on all four cards at the **full native 262,144 ctx** (fits in 54 of 64 GiB; the 131k limit was a pair budget, not a model limit) - Attachable build sessions: `OPENCODE_ATTACH_PORT=4096` publishes the seat; `GET /build` carries the per-session password — `opencode attach http://trogdor:4096` and you watch the developer live - Phaser skills symlinked into OpenCode's **native skill discovery**, so sessions with no AGENTS.md still see them; searxng MCP wired into the seat (per-session); **rexUI** (MIT, official Phaser 4 port, pinned) named trusted in AGENTS.md - Correction for the record: an earlier claim here that the Phaser skills were never wired was false — a grep filtered lines containing "kill", which excludes "skill". They were vendored, pinned, and guarded all along. Stack env restored to trogdor-speed. Play the corpse: http://trogdor:3923/games/bakeoff-baseline/index.html · the comparison page has its card.
Author
Owner

baseline-eyes: one tool back, and the corpse stands up

Session of 2026-07-29. The ablation ladder is complete — same agent, same brief, same materials, only the feedback varies:

arm feedback result time
naked baseline none black screen (one v3-ism, never seen) 7 min
baseline-eyes v1 playtest numbers (console, errors, motion) plays, verified 9.5 min
baseline-eyes v3 numbers + frames as MCP images plays, verified, full scene: sky gradient, skyline, HUD, lives, level/speed 23 min, 9+ playtests
round-3 editor (process) full pipeline plays, verified, real pixel art, one palette ~45 min end to end

Play them: eyes · naked · process · cards

Findings

  1. Feedback, not capability, was the gap — confirmed twice. Numbers alone flipped the black screen to a playing game; frames turned it into a composed one. The prompt change mattered too: "done = an OBSERVED passing playtest" gave the loop a termination condition the naked arm never had.
  2. playtest_build now hands back pixels (1ee8f13, after 93bc1fa): mid-play + final frame as MCP image content beside the JSON report. Two live-caught bugs on the way: the agent trying to SEE its screenshots through a text extractor (why the feature exists), and the tool's own -> dict annotation rejecting the mixed reply while the agent read the report out of the pydantic error text and iterated anyway (why the annotation is now Any).
  3. Where the process still wins: art. The eyes game's graphics are soft programmatic shapes; the process game is actual pixel art with a palette. The baseline ladder measured engineering feedback; craft is still the pipeline's moat — which is the right division of labour.
  4. Ops that landed: attach hints now carry -s <session> (user suggestion) so the TUI opens straight on the live session.

Next

  • LSP round (queued): typescript-language-server + phaser.d.ts pinned beside the skills — edit-time diagnostics for the invented-API class; needs the code-in-.js convention nudge.
  • Studio build-session pane (queued) — the ticker earned a real view.
  • Stack env restored to trogdor-speed.
## baseline-eyes: one tool back, and the corpse stands up Session of 2026-07-29. The ablation ladder is complete — same agent, same brief, same materials, only the feedback varies: | arm | feedback | result | time | |---|---|---|---| | naked baseline | none | **black screen** (one v3-ism, never seen) | 7 min | | baseline-eyes v1 | playtest numbers (console, errors, motion) | **plays, verified** | 9.5 min | | baseline-eyes v3 | numbers + **frames as MCP images** | **plays, verified**, full scene: sky gradient, skyline, HUD, lives, level/speed | 23 min, 9+ playtests | | round-3 editor (process) | full pipeline | plays, verified, real pixel art, one palette | ~45 min end to end | Play them: [eyes](http://trogdor:3923/games/bakeoff-baseline-eyes/index.html) · [naked](http://trogdor:3923/games/bakeoff-baseline/index.html) · [process](http://trogdor:3923/games/bakeoff-editor/index.html) · [cards](http://trogdor:3923/games/bakeoff/index.html) ### Findings 1. **Feedback, not capability, was the gap — confirmed twice.** Numbers alone flipped the black screen to a playing game; frames turned it into a composed one. The prompt change mattered too: "done = an OBSERVED passing playtest" gave the loop a termination condition the naked arm never had. 2. **`playtest_build` now hands back pixels** (`1ee8f13`, after `93bc1fa`): mid-play + final frame as MCP image content beside the JSON report. Two live-caught bugs on the way: the agent trying to SEE its screenshots through a text extractor (why the feature exists), and the tool's own `-> dict` annotation rejecting the mixed reply while the agent read the report out of the pydantic error text and iterated anyway (why the annotation is now `Any`). 3. **Where the process still wins:** art. The eyes game's graphics are soft programmatic shapes; the process game is actual pixel art with a palette. The baseline ladder measured *engineering* feedback; craft is still the pipeline's moat — which is the right division of labour. 4. Ops that landed: attach hints now carry `-s <session>` (user suggestion) so the TUI opens straight on the live session. ### Next - **LSP round (queued):** typescript-language-server + phaser.d.ts pinned beside the skills — edit-time diagnostics for the invented-API class; needs the code-in-.js convention nudge. - Studio build-session pane (queued) — the ticker earned a real view. - Stack env restored to trogdor-speed.
Author
Owner

baseline-pixel: the merge arm ran end to end, and the loops all closed

Session of 2026-07-29 (late). The eyes loop + the two pixel-art tools, shaped by the user: the AGENT is the review loop — draw returns the first version, the agent reads what it looks like, tweaks by instruction. Getting here burned four runs on infrastructure walls, each now fixed and tested:

  1. MCP client cancelled slow drawsexperimental.mcp_timeout (600s)
  2. Batched draws died queueing on the brain's one slot → one-draw-at-a-time lock whose instant refusal teaches the workflow (5e7a045…)
  3. LSPs are opt-in in OpenCode → explicit lsp block; typescript-language-server + Phaser 4's own phaser.d.ts (pinned) + per-workspace jsconfig
  4. The wire-format wall: OpenAI-compatible chat has NO image slot in tool results — no flag can ever deliver tool-result pixels to this brain (attachment: true fixed the wrong layer). The looking moved server-side: the vision brain captions every draw/tweak (looks_like) and every playtest's mid-play frame (frames_show), and the caption rides the text channel the model actually receives.

The run (83 min, 19 draws, 9 playtests, zero tool errors)

The caption loop worked immediately and repeatedly: "the dusk sky came out looking like a planet — let me fix that", a catch-pose anatomy fix, "the battery looks like a battery", and frames_show catching a missing-texture green square in the running game. The one-palette guidance + workspace palette anchor produced the most cohesive scene of any run: purple/orange throughout, starry dusk, dithered building texture, brick rooftop.

result
plays TRUE — 59.1 fps, motion 7.81 (highest yet)
art 11 committed pixel-art assets, all drawn via the tools
verified FALSE — assets_used: three superseded background DRAFTS committed but unused (iteration debris; the finals are wired). Bare arm gets no repair pass by design
the miss the robot renders ~5x its intended size — a display-scale bug neither the agent nor frames_show called out

Play it: http://trogdor:3923/games/bakeoff-baseline-pixel/index.html

Next levers, in order

  1. Scale discipline: the pixelArt/integer-zoom rule made pixels crisp but nothing polices display size vs authored size — a one-line rule (setDisplaySize = authored size × integer zoom) in the task/AGENTS.md, and frames_show should be asked to compare subject scale against the scene.
  2. Draft hygiene: superseded draws should be deleted or the task should say to clean up — cheap wording fix; or bare arms could allow the one repair pass.
  3. DISABLED_TOOLS naming never matched (the agent saw generate_asset and got gate-refused instead) — harmless with the gate, worth fixing for cleanliness.
  4. Animation#39.

Stack env restored to trogdor-speed. All fixes deployed and permanent.

## baseline-pixel: the merge arm ran end to end, and the loops all closed Session of 2026-07-29 (late). The eyes loop + the two pixel-art tools, shaped by the user: the AGENT is the review loop — draw returns the first version, the agent reads what it looks like, tweaks by instruction. Getting here burned four runs on infrastructure walls, each now fixed and tested: 1. **MCP client cancelled slow draws** → `experimental.mcp_timeout` (600s) 2. **Batched draws died queueing on the brain's one slot** → one-draw-at-a-time lock whose instant refusal teaches the workflow (`5e7a045`…) 3. **LSPs are opt-in in OpenCode** → explicit `lsp` block; typescript-language-server + Phaser 4's own `phaser.d.ts` (pinned) + per-workspace jsconfig 4. **The wire-format wall**: OpenAI-compatible chat has NO image slot in tool results — no flag can ever deliver tool-result pixels to this brain (`attachment: true` fixed the wrong layer). The looking moved server-side: the vision brain captions every draw/tweak (`looks_like`) and every playtest's mid-play frame (`frames_show`), and the caption rides the text channel the model actually receives. ### The run (83 min, 19 draws, 9 playtests, zero tool errors) The caption loop worked immediately and repeatedly: *"the dusk sky came out looking like a planet — let me fix that"*, a catch-pose anatomy fix, *"the battery looks like a battery"*, and `frames_show` catching a missing-texture green square in the running game. The one-palette guidance + workspace palette anchor produced **the most cohesive scene of any run**: purple/orange throughout, starry dusk, dithered building texture, brick rooftop. | | result | |---|---| | plays | TRUE — 59.1 fps, motion 7.81 (highest yet) | | art | 11 committed pixel-art assets, all drawn via the tools | | verified | FALSE — `assets_used`: three superseded background DRAFTS committed but unused (iteration debris; the finals are wired). Bare arm gets no repair pass by design | | the miss | **the robot renders ~5x its intended size** — a display-scale bug neither the agent nor `frames_show` called out | Play it: http://trogdor:3923/games/bakeoff-baseline-pixel/index.html ### Next levers, in order 1. **Scale discipline**: the pixelArt/integer-zoom rule made pixels crisp but nothing polices *display size vs authored size* — a one-line rule (`setDisplaySize` = authored size × integer zoom) in the task/AGENTS.md, and `frames_show` should be asked to compare subject scale against the scene. 2. **Draft hygiene**: superseded draws should be deleted or the task should say to clean up — cheap wording fix; or bare arms could allow the one repair pass. 3. **DISABLED_TOOLS naming** never matched (the agent saw `generate_asset` and got gate-refused instead) — harmless with the gate, worth fixing for cleanliness. 4. **Animation** — #39. Stack env restored to trogdor-speed. All fixes deployed and permanent.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
cmoriarty/trog#38
No description provided.