Pixel art by placement: build the editor-MCP arm, and go learn the craft #38
Labels
No labels
bug
duplicate
enhancement
help wanted
invalid
question
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
cmoriarty/trog#38
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Why now
#37's bake-off settled the direction. Same fixture, same brief, same grid,
same developer agent — only the generator differed:
Arm C's robot is a robot — symmetrical chassis, cyan eye panels, antennae,
shoulder pads, panelled torso, one consistent palette, clean outline,
filling its frame. The incumbent's is a fragment blob. Play them:
http://trogdor:3923/games/bakeoff/index.html
Arm C was deliberately the cheap proxy for the editor-MCP hypothesis —
deliberate pixel placement beats sampling-then-downscaling — built with a
6-op renderer and no dependencies precisely so the real investment would
only be made if the hypothesis held. It held. So build the real thing.
The other half of this ticket matters as much: arm C wins on placement
and still loses on craft. Its battery reads as a canister because it
dropped the yellow terminal cap the brief asked for. It has no notion of
ramps, dithering, sub-pixel weight, or how a real artist decides what to
cut at 32px. That knowledge exists; we have never gone and got it.
Part 1 — Arm B: a real pixel editor as an authoring seat
Candidates (already surveyed in
docs/toolbelt-audit.md, scoped there aspost-processing; this reframes them as authoring):
animation, retro palettes, dithering, shading, spritesheet export
Licensing decides the shape: Aseprite's binary is paid and its source
non-free (self-compile, personal use). LibreSprite
is the GPL fork and scripts headlessly in Lua or JS. Either way it is
CPU-only, deterministic, and does not contend with the brain for GPUs —
which is why it fits the seat model.
opencode/andplaytest/use)ARMS["editor"]introg_lib/bakeoff.pyis onedict entry — the harness already holds everything else constant
so if adopted it belongs in common-mcps with its own stack, the
way searxng does — not in trog compose
What would make it beat arm C: real dithering and ramp tools, layers,
sheet export, and animation frames — the things a 6-op renderer cannot do.
If it does not beat arm C, say so and keep the cheaper thing.
Part 2 — go and learn the craft, then bake it in
docs/pixel-art-craft.mdexists (#20, from saint11/Lospec) and is wiredinto the critic as a style-block opt-in. It is a rubric for JUDGING. What we
do not have is craft knowledge shaping how art gets AUTHORED — which is now
the binding constraint, because arm C proves we can place any pixel we like
and the open question is which pixels.
Research targets:
readability at the played size, what gets cut at 32px vs 64px, why
the eye reads a shape in a quarter second. saint11, Pedro Medeiros,
Lospec tutorials, MortMort, Adam Saltsman's writing.
highlights toward warm), why a 16-colour game palette beats
per-asset palettes, how Celeste/Dead Cells/Stardew constrain
themselves. We now enforce one palette per game mechanically
(
8ba4c3b) — the open question is how to CHOOSE it well.readable at speed), Stardew (warm consistent light, dense but legible
tiles), Terraria (huge asset count, one coherent look), Dead Cells
(3D-rendered then hand-retouched into pixel art — a genuinely
different pipeline worth understanding, and possibly a third arm).
convert to sprites. If that is tractable it sidesteps both the
diffusion and the placement problem for ANIMATION, which is our
weakest area.
anticipation. pix3lwalk covers walk cycles only; #20 found nothing
purpose-built for object motion.
Output, and this is the part that must not be skipped: every finding
lands as something mechanical, not as advice in a document. #20's
research was correct and evaporated because its conclusion was addressed to
a human who is not in an unattended run. So each lesson becomes one of:
a standing prompt suffix, a
pixelprogop or system-prompt rule, adeterministic gate in
critics.cutout_problem, or a floor inassets.py.If a finding cannot be made mechanical, it does not count as done — write
down explicitly who is supposed to act on it and when.
Part 3 — settle why the incumbent was so bad
Not to rescue it, but because we do not understand our own tools and that
is its own risk. Leading suspect: the cutout is eating dark subjects.
Arm A's robot came back 3 colours at 3.1% coverage on a dark navy dusk
palette;
cutout_backgroundflood-fills from the border throughsame-coloured pixels, and a slate robot against a dark backdrop is exactly
where subject and background quantise close enough to merge. The bright red
heart survived; the dark robot did not.
and/or force a contrasting backdrop in the prompt
16x to a 32 grid; PixelOE contrast downscale on dark low-contrast
art; the craft suffix's "centered on a plain background" fighting
"fill the frame"
docs/asset-gen-research.md) and the revisit has still not happenedBar
Same as always: open weights or self-hostable, commercially usable licence,
no phone-home. Hosted pixel-art SaaS (PixelLab,
Retro Diffusion) stays excluded for the same
reason the Phaser Game Agent MCP did — worth ONE manual run as a reference
ceiling, so we know what we are giving up.
Judged the way #37 judged: by playing the game, not by looking at a sprite
sheet. The harness is
trog test bakeoff.Part 1 decided and built; first editor-arm bake-off run is in
Session of 2026-07-28 (evening). Commits
d3c159f,a7d23a8— pushed, CI green, deployed.The licence survey settled the editor pick, and the pick is: no binary
aseprite_pathfor all functionality (no LibreSprite mode).pixelprogalready does, with none of the dithering/shading/ramp tools that were the whole argument for an editor (those are post-fork Aseprite features). Its MCP wrappers attach to a running GUI over a port; "hacky and brittle" is their own README's phrase.So the real thing the winning hypothesis justified is the editor's toolset, not an editor binary:
trog_lib/pixeledit.py, armeditorin the bake-off. Flood fill, deliberate dither (3 named patterns), directional auto-shade, selective outline — plus part 2's craft rules enforced, not advised (the #20 lesson, applied):shade/seloutderive every highlight from it, so pillow shading cannot come out of the opdocs/pixel-art-craft.mdnow carries the rule→mechanism table. The findings that could not be made mechanical (game-palette choice, animation frames, the Dead Cells 3D→sprite pipeline as a possible third arm) are listed there with a named owner and trigger each — not as advice.docs/toolbelt-audit.md's aseprite row is now rejected with the reasoning. willibrandon/pixel-mcp (MIT) stays a reference for dither/shade algorithms.First run:
trog test bakeoff --arms commands,editorassets_used: committed but never loadedPlay them (the verdict, per the house rule):
assets_usedcaught it deterministically.Read of the run: the editor arm went 4-for-4 through the new gates (zero unusable programs), and its craft mechanics all fired — the robot has a consistent top-left light, hue-shifted ramps on the limbs, and no dead uniform outline; it is the best sprite either program arm has produced. By eye the generator comparison is otherwise mixed (control's battery is more readable; both hearts are weak; both backgrounds are primitive gradient-bands — composition is a different problem from craft ops). The control losing on
assets_usedis build-session variance, not generator quality — worth remembering when reading the verified column.Open, in order
shade+ ramps made the editor's sprites read muddier than the control's bold flats at 1x. Candidate mechanism: a value-structure rule (big value separation between adjacent forms) in the prompt, possibly a deterministic palette value-spread check.Brain bake-off: four hands, one pencil — and the biggest hand did not win
Session of 2026-07-28 (night). Same generator (
pixeledit), same fixture, same developer (BUILD_MODEL = qwen3.6-35b-a3b) — only the model writing the drawing programs varied. Commitsc0b67ff(harness:--modelsmatrix, per-model repos, programs committed beside PNGs, per-asset gallery),74a7516(gate fix, below). All deployed, CI green, stack env restored to trogdor-speed.Compare + play all four (per-asset gallery with coverage/ops/ramps/draw-time under each sprite):
http://trogdor:3923/games/bakeoff/index.html
The eye's read
What actually separated the models: the craft gates
The strict dialect is now the dominant failure mode, and it discriminates by model habit:
ramp never shifts hue, and it did not correct from the error text within the 2-try budget. 122B lost two the same way; the baseline (which grew up on this dialect) lost none.#00000000— the RGBA idiom for transparent) and the parser only takes 6.Round-3 dialect changes (apply to ALL models equally, never mid-experiment): accept 8-digit hex and fold alpha into the transparency contract; put a concrete example ramp in the hue-shift error; consider 3 tries; possibly auto-repair a flat ramp (mechanically bend shadows cool / highlights warm) instead of rejecting.
Infra incident, root-caused and fixed
The 122B build's developer called
generate_assetmid-session. The /jobs gate keyed on_build_task— but a bake-off arm's build runs inside_bakeoff_task— so the request passed, the image seat took the brain's cards under the non-resident brains profile, and the agent's own backend died under it. Build stalled 600s; the next arm's draws failed on connection errors until the seat TTL restored the brain. Fixed in74a7516:codeseat.STATE.status == "running"is the fact the gate reads now, whatever task wraps the session. Gemma got a clean re-run after the fix (plays + verified); the page carries the merged results.The other standing result
assets_usedfailed on 2 of 4 builds — the shared developer wires committed art into the game roughly half the time, independent of who drew it. That is now the biggest non-art quality gate. Candidate: one repair session keyed on a failedassets_usedcheck, inside the existingBUILD_FIX_RETRIESbudget.Bigger models, round 2 (queued, not run)
DeepSeek-V4-Flash and Hy3 were downloaded and then dropped (no vision — user call); weights deleted. Qwen3.5-397B-A17B UD-Q3_K_XL (179G, vision-capable) is on the RAID awaiting a roster entry with CPU expert offload (
-ncmoe) — it cannot fit 64GB VRAM.Playable demos
Round 3: the editor got eyes, and the background got a skyline
Session of 2026-07-29 (early). Commits
28bdbb1(draw–look–revise, gate auto-repair, per-model coding, assets_used repair session),a6f787f(revision no-regression guard). Deployed, CI green, run under plain trogdor-speed with the resident 35B-A3B.Play it: http://trogdor:3923/games/bakeoff-editor/index.html · gallery: http://trogdor:3923/games/bakeoff/index.html
What changed
PIXELEDIT_REVISIONS(3) looks. KEEP ends early.codeseat.build(model=)).assets_usedrepair session when a build ships art it never loads.What the first unguarded run taught (and the guard it forced)
The model answers "revise" with NEAR-EMPTY programs despite being asked for complete ones: a 100% background fell to 2%, the battery to 30%, the heart to 11%.
a6f787f: a revision that halves coverage is refused, costs its slot, and the refusal is worded into the next look. The guarded rerun immediately behaved.The guarded run
Build shipped dead art → the repair session fired for the first time and fixed it: plays, verified, every asset loaded. That closes the loop nothing else had closed — art reaching the screen is now enforced, not hoped for.
The background is the story. For the first time in any run of any arm it is a composition: banded dusk sky fading indigo→orange, cloud silhouettes, stars, a building skyline, deliberate dither texture on the warm band and the rooftop, one detail cluster (an AC unit) off-centre. That is the composition prompt + the dither op + three looks at its own render, compounding. Two critic passes on sprites is also the best bare-run showing. The heart finally has lobes, hue-shifted shading and a highlight — its coverage (31%) is honest for the shape.
The critic still fails the background; on the evidence of the in-play frame the critic is now behind the eye on backgrounds — calibration note for #18's successor, not a gate change tonight.
Standing
The naked baseline: seven minutes, 733 lines, dead on arrival
Session of 2026-07-29. The null hypothesis the whole studio has to beat (commits
5237ab4,85282f6,1df8a53): one agent, the brief inline, Phaser 4 skills + rexUI + web search as materials, the Q8/262ktrogdor-coderbrain, no process — no design docs, no critics, no revise, no repair, no playtest tool, no time cap. Judged after the fact by the same recorded playtest as every arm.Result
Graphicstextures (its own answer to having no asset pipeline)scene.make.graphics({add:false})— a Phaser 3 idiom, line 28, first call of its texture pipeline →Cannot read properties of undefined (reading 'sys')→ one uncaught error kills all 733 linesThe agent read the skills (native skill tool fired — the new discovery symlinks work), wrote a complete, confident, well-structured game around ONE v3-ism it never saw fail — because the one thing it lacked was any way to RUN the game. It did not use rexUI; it did not search the web; it built exactly what it believed and shipped it blind. AGENTS.md's black-screen lore, reproduced from scratch on the first try.
Reading
playtest_build(ours). A coding agent without execution feedback ships confident corpses; with it, the same model iterates to a playing game every time it has run under the harness.Infra that landed with this (all deployed)
trogdor-coderprofile/roster: 35B-A3B Q8 on all four cards at the full native 262,144 ctx (fits in 54 of 64 GiB; the 131k limit was a pair budget, not a model limit)OPENCODE_ATTACH_PORT=4096publishes the seat;GET /buildcarries the per-session password —opencode attach http://trogdor:4096and you watch the developer liveStack env restored to trogdor-speed. Play the corpse: http://trogdor:3923/games/bakeoff-baseline/index.html · the comparison page has its card.
baseline-eyes: one tool back, and the corpse stands up
Session of 2026-07-29. The ablation ladder is complete — same agent, same brief, same materials, only the feedback varies:
Play them: eyes · naked · process · cards
Findings
playtest_buildnow hands back pixels (1ee8f13, after93bc1fa): mid-play + final frame as MCP image content beside the JSON report. Two live-caught bugs on the way: the agent trying to SEE its screenshots through a text extractor (why the feature exists), and the tool's own-> dictannotation rejecting the mixed reply while the agent read the report out of the pydantic error text and iterated anyway (why the annotation is nowAny).-s <session>(user suggestion) so the TUI opens straight on the live session.Next
baseline-pixel: the merge arm ran end to end, and the loops all closed
Session of 2026-07-29 (late). The eyes loop + the two pixel-art tools, shaped by the user: the AGENT is the review loop — draw returns the first version, the agent reads what it looks like, tweaks by instruction. Getting here burned four runs on infrastructure walls, each now fixed and tested:
experimental.mcp_timeout(600s)5e7a045…)lspblock; typescript-language-server + Phaser 4's ownphaser.d.ts(pinned) + per-workspace jsconfigattachment: truefixed the wrong layer). The looking moved server-side: the vision brain captions every draw/tweak (looks_like) and every playtest's mid-play frame (frames_show), and the caption rides the text channel the model actually receives.The run (83 min, 19 draws, 9 playtests, zero tool errors)
The caption loop worked immediately and repeatedly: "the dusk sky came out looking like a planet — let me fix that", a catch-pose anatomy fix, "the battery looks like a battery", and
frames_showcatching a missing-texture green square in the running game. The one-palette guidance + workspace palette anchor produced the most cohesive scene of any run: purple/orange throughout, starry dusk, dithered building texture, brick rooftop.assets_used: three superseded background DRAFTS committed but unused (iteration debris; the finals are wired). Bare arm gets no repair pass by designframes_showcalled outPlay it: http://trogdor:3923/games/bakeoff-baseline-pixel/index.html
Next levers, in order
setDisplaySize= authored size × integer zoom) in the task/AGENTS.md, andframes_showshould be asked to compare subject scale against the scene.generate_assetand got gate-refused instead) — harmless with the gate, worth fixing for cleanliness.Stack env restored to trogdor-speed. All fixes deployed and permanent.