| LICENSE | ||
| README.md | ||
LLM Pixel Artist
A methodical exploration of one question: can an LLM make pixel art without the need of image generation?
Whether the output would survive being looked at by someone who knows what they're looking at.
1. Style Focus
To focus on a particular style, the target style is Celeste-class: true hand-placed pixel art on a small canvas, at a low internal render resolution, read in motion at speed.
What "Celeste-class" actually commits us to:
- A small canvas where every pixel is a decision. A 32×32 sprite is 1024 decisions. That is a search space small enough to reason about explicitly, which is exactly why this problem is tractable for a language model at all.
- A low internal resolution the whole scene shares. Celeste renders internally at 320×180 and upscales. Every asset therefore lives on one pixel grid at one density — this is not a per-asset choice, it is a project-level constant.
- Visual inspection should include the game context. A sprite that only looks good at 8× zoom is a failed sprite. The judgement that matters is at play size, at play speed, over the actual background.
- Silhouette/Outline over internaldetail. At this size interior detail cannot rescue a mushy outline.
Tasks for Style Focus
Before generating anything, measure and write down:
- The internal resolution and pixels-per-unit we are targeting.
- Character height in pixels, and every other asset's scale relative to it.
- Canvas sizes for each asset class (character / prop / tile / icon / UI).
- Frame counts for each motion class.
2. Lessons learned from Trog
Rejection is a worse teacher than repair. Refusing a malformed ramp zeroed five real assets across three models, because "same hue, darkened" is how most models write their first ramp and two retries is not enough to unlearn it. Mechanically repairing it and disclosing the repair so the model can overrule it worked. Prefer repair-and-disclose to refuse-and-retry.
Drawing blind is the biggest single gap. One shot, no look at the result, with a parse error as the only feedback channel. An artist's loop is draw → step back → fix. Any loop without a look at the render is not a drawing loop.
Formatting habits are not craft failures. Refusing #00000000 — the
standard RGBA idiom for transparent — cost a whole asset. Be liberal about
syntax, strict about craft.
Walls to Climb
Design constraints for this one, stated as things to get right from the start:
| Previous ceiling | What it cost | Requirement here |
|---|---|---|
| Flat canvas, no layers | Animation impossible; no way to move a limb without redrawing the sprite | Layers are in the model from day one |
| No history / no undo | Every revision was a full rewrite from memory, so each round drifted | Revision is a diff against a held canvas, not a re-emission |
| Malformed drawing operations silently skipped | The model never learned it drew something wrong | Every rejected drawing operation is reported back; silence is a quality leak |
| Out-of-range colour indices clamped to nearest | Silently wrong colours that look like an art failure | Out-of-range is an error with a message, not a guess |
3. Method
The phases are ordered so that each one is measurable when it starts.
Phase 0 — Ground truth first
Curate a small set of pixel art we are willing to call good.
Everything downstream calibrates against it: the rubric, the linter, the visual critic, and eventually the models. Without it, "the LLM judge is unreliable" and "the linter is too strict" are both unfalsifiable complaints.
Build:
- A gold set — a handful of assets, at the target grid: an icon, a character idle, a tile, a prop. Small. It does not need to be beautiful; it needs to be honestly judged and documented as to why.
- The rubric — in addition to passing on its own merit, it also needs to be proven to pass a visual test in it's game enviornment (both still and animated).
Phase 1 — Make quality measurable
Build the judging apparatus, which initially will focus on human visual judgement. This implies for any experiment that a test harness generate an interactive visual report/questionnaire that allows a human to judge the model output with metadata and context about each judgment or decision the human needs to make. Building a programmatic/LLM-based judging will be a challenge, but if we store the human judgements and reasoning, perhaps we can create a set of criteria for judging model output.
Phase 2 — Visualizing pixel art within game setting
- Palette incoherence — assets generated independently have independent colours. A game reads as amateur the instant its palette is not shared.
- Inconsistent pixel density — a 32×32 sprite beside a 64×64 sprite at the same on-screen size. The most common tell of AI-generated "pixel art."
- Scale incoherence — a 16px character beside a 40px door.
- No motion — game feel is animation, not illustration.
So: palette and style guide become project artifacts, generated once up front, injected into every request, gives critics a reference point for judging.
- Palette (16–32 colours) with explicit ramps, hue-shifted — shadows toward blue/purple, highlights toward yellow/orange. Value-only ramps are the fastest route to flat and amateur. Choosing the palette well is itself an open problem; seeding from a curated ramp-complete palette near the brief's mood is one candidate mechanism.
- Style guide — canvas sizes, character height, pixels-per-unit, light direction, outline rule (selective outlining in a darker shade of the adjacent colour, not universal black), dithering rules, ground-line convention.
Evaluate at set level and on-background from here on. A screenshot, not a sprite sheet.
Phase 3 — Make it move
Redrawing each frame from scratch guarantees inconsistency and costs N× the tokens. This is why layers are required from day one.
- Draw a base pose; define frames as transformations of layered body parts (translate / rotate / scale per layer), with in-betweens generated programmatically and retouched.
- Onion-skin the previous and next frames into the critique image.
- Encode the game-feel vocabulary explicitly: squash and stretch, anticipation, smear frames, a held contact frame. These are the difference between "animated" and "alive."
- Target frame counts deliberately, from the style guide.
4. Open decisions
- Backend. Which models play artist and which play critic, and whether they are hosted or API. Phase 2 deliberately uses the strongest available option to find the ceiling; the production choice is a separate, later question informed by the harness.
- Canvas / project representation. Layers are required; the concrete format (in-memory model, on-disk serialization, whether animation structure is first-class) is undecided and should be settled at the start of Phase 2, since representation is one of that phase's variables.
- Where the gold set comes from. Hand-made by us is the default and keeps the repo clean of shipped game assets. How much, and how good it needs to be, is open.
5. Prior art
trog (server/trog_lib/pixeledit.py, pixelprog.py, pixel_post.py,
docs/pixel-art-craft.md) is the first attempt, and stays frozen as a control
to measure against. Its craft checklist, its op dialect, and above all its
recorded failures.
The craft vocabulary itself is the field's consensus, not one artist's taste — Pedro Medeiros' (saint11) tutorial corpus at https://saint11.art/pixel_articles/, written by an artist who worked on Celeste and TowerFall, and Lospec's tutorial library and palette discipline at https://lospec.com/pixel-art-tutorials.