Agentic Game Development Studio

A fully self-hosted, agent-driven game studio on one server.

What it is

A local game studio where one AI model (driven by OpenCode) does the heavy lifting across planning, code, art, music, sound, and playtesting — brief in, finished LÖVE game out. Four principles:

1. One seat, all the resources. A single multimodal model (the studio seat) authors the design docs, generates + QAs the assets, writes the game, AND serves the review judge — no second model to hand off to. LangGraph provides the guardrails the old two-seat "director" was built for.
2. Verification lives in tools, not prompts. The gateway's check_phase gate, deterministic validators, the commit-keyed tool ledger, and running the game (run_love2d) are the real gates. A claim is never verification until a tool or check_phase says so.
3. Every generator has a critic. Images are judged by the seat's own vision tower, music is scored against its brief, games are run and their screenshots reviewed.
4. One compose stack, git is truth. One docker-compose.yml, deployed as a Portainer git stack from Forgejo.

flowchart TD
    You["👤 You<br/>(brief in, game out)"]
    Pipe["⚙️ Studio pipeline<br/>laptop LangGraph scheduler<br/>fresh session per phase/milestone"]
    Check["🚦 check_phase gate<br/>gateway-side, deterministic<br/>tool ledger keyed by commit SHA"]
    Seat["🧠 Studio seat · Qwen3.6-27B<br/>262K ctx · 35.6 t/s · vision ON<br/>designs · generates · codes · self-critiques"]
    Zoo["🧰 The Zoo<br/>image · music · SFX · TTS · playtest · CC0 library"]
    Batch["📦 Batch conductor<br/>brain-down model swap for big generators<br/>job queue + per-profile deploy"]

    You -->|"brief"| Pipe
    Pipe -->|"phase spec + pointers"| Seat
    Seat -->|"MCP tool calls"| Zoo
    Seat -->|"commit + push"| Check
    Check -->|"green / findings"| Pipe
    Pipe -->|"submit manifests"| Batch
    Batch -->|"swaps generator onto borrowed GPUs"| Zoo
    Pipe -->|"finished game"| You

---

Set up from scratch

Two machines: Trogdor (the GPU server — Docker + NVIDIA container toolkit + a Portainer agent) and a laptop (drives everything over SSH). Self-hosted Forgejo provides git + the container registry.

1. Clone + configure.

   git clone https://forgejo.underthere.xyz/cmoriarty/agentic-game-dev.git
   cd agentic-game-dev
   cp .env.example .env    # then edit: REGISTRY, MODELS_ROOT, OUTPUT_ROOT,
                           # ASSETS_ROOT, PORTAINER_WEBHOOK (step 5), COMPOSE_PROJECT
   

   Compose ${VAR:-default} fallbacks are the source of truth for model config — .env and Portainer env vars carry only deliberate divergences.

2. SSH + registry. Passwordless SSH to cmoriarty@trogdor (every make target uses DOCKER_HOST=ssh://…), then make login for the Forgejo registry.

3. Models onto the RAID (hf CLI with HF_HUB_ENABLE_HF_TRANSFER=1 — ~86 MB/s vs single-stream curl; download by explicit filename, the --include flag silently does nothing):

   # The seat (Qwen3.6-27B, dense, ~38 GB on disk):
   cd /mnt/models/Qwen3.6-27B-GGUF
   HF_HUB_ENABLE_HF_TRANSFER=1 hf download unsloth/Qwen3.6-27B-GGUF \
     Qwen3.6-27B.gguf --local-dir .
   # Vision tower + mmproj (mmproj BF16 for gemma fallback is the same field):
   HF_HUB_ENABLE_HF_TRANSFER=1 hf download unsloth/Qwen3.6-27B-GGUF \
     mmproj-BF16.gguf --local-dir .
   

   The Qwen3.6 chat template handles opencode's multiple system messages natively — no patch needed (the scripts/chat_template_multisys.jinja patch from earlier MoE seats is no longer required). ComfyUI checkpoints and audio model caches live under $MODELS_ROOT/comfyui/ and $MODELS_ROOT/audio/ (see .env.example for the layout; audio weights self-download on first use).

4. CC0 asset library (once, on Trogdor): the scripts/asset-library/download_*.py mirrors (Kenney, game-icons, ambientCG, OGA) populate $ASSETS_ROOT/library, then index_qdrant.py embeds it into Qdrant.

5. Portainer git stack. In Portainer: Stacks → Add → Repository, point at this repo's docker-compose.yml on Forgejo, leave the stack env empty, copy the stack's webhook URL into .env as PORTAINER_WEBHOOK (add PORTAINER_INSECURE=1 for a self-signed cert, COMPOSE_PROJECT=<stack name> so make ps/logs target the right containers).

6. Build + deploy.

   make deploy   # build custom images on Trogdor → push to Forgejo → webhook redeploy
   make warm     # first request pays a one-time ~2 min CUDA init — pay it now
   make smoke    # one-screen ✅/❌ tree of every service and model
   

7. Laptop python env (the pipeline + tests run here, no GPU needed):

   make test-install   # pip install -r requirements-dev.txt (includes `studio` CLI)
   make test           # 232 laptop tests, no Docker/weights required
   

8. Wire OpenCode: make opencode-setup — symlinks AGENTS.md, the studio agent, plugins, and commands into ~/.config/opencode/ (game repos live outside this repo, so config must be global), prunes dead links, and verifies the provider/MCP blocks and both endpoints.

Host tuning that matters: kernel.numa_balancing=0 (/etc/sysctl.d/99-numa-balancing.conf on Trogdor) — kernel page migration hurts CPU-side MoE offload during brain-down batch swaps.

---

Start a new game

Autonomous (the default path — brief in, game out):

make studio-run BRIEF="a cozy fishing game with a day/night cycle" DIR=~/games/fishing
# long / quoted briefs:
make studio-run BRIEF_FILE=brief.md DIR=~/games/fishing

The pipeline preflights the wiring (starts opencode serve itself if needed), creates the game repo + Forgejo remote, then drives design phases 0–5 and the milestone build loop with no human handoff. Watch it:

make studio-status DIR=~/games/fishing   # step, verdict, attempt, history
make logs S=mcp                          # gateway: check_phase + batch activity
make studio-up DIR=~/games/fishing       # then: opencode --hostname 127.0.0.1 --port 4096
                                         # attaches a TUI to the live seat session

Manual (you drive the same process turn by turn): open opencode in an empty game folder and run /new-game <brief> on the studio agent. Same doc templates, same check_phase gates, same commit-keyed ledger.

Both paths produce a Forgejo-backed repo: docs/01–04, an assets/ library, LÖVE source, tests.

Deadlines are env-tunable (design turns default 45 min, asset/build turns 90 min): STUDIO_MILESTONE_DEADLINE_S=10800 make studio-run …. STUDIO_MAX_ATTEMPTS (default 3) caps retries per step before the run parks itself blocked.

Resume an interrupted run

State is durable — every step checkpoints to ~/.local/state/trogdor-studio/pipeline.sqlite (one thread per game), so a laptop reboot, a crash, or a killed terminal loses nothing.

make studio-status DIR=~/games/fishing        # where did it stop?
make studio-resume DIR=~/games/fishing        # default ACTION=retry
make studio-resume DIR=~/games/fishing ACTION=skip   # force-advance past the step
make studio-resume DIR=~/games/fishing ACTION=abort  # end the run

Three interruption shapes, all handled by resume:

• Blocked — a step failed MAX_ATTEMPTS times; status shows the step, notes, and history. retry resets the attempt counter (the rejection notes carry into the next turn), skip force-advances, abort ends it.
• Crashed / killed mid-turn — resume re-executes the pending node. Phases re-verify the pushed repo state first (check_phase), so already-satisfied work is recognized, not redone.
• Deliberate pause — just Ctrl-C the run; resume later.

Update an existing game

Two ways, depending on how much process you want:

Manual (quick changes, tweaks, new content): open opencode in the game dir — the committed docs/ are the handoff state; the seat re-reads them. Prompt the change, let it use the zoo, then /playtest [seconds] to verify (commits, pushes, runs headless LÖVE, critiques the screenshots against the storyboard). Commit + push is part of done.

Pipeline-driven (a scoped work package): append milestones to docs/04-devplan.md (same ### Milestone N format — Scope + Done-when bullets), commit + push, then start a fresh pipeline thread over the same repo:

studio run "add a shop and currency system" --dir ~/games/fishing --game fishing-update1

A finished thread is terminal, so the new --game id gives the update its own thread; phases 0–5 re-verify green against the pushed repo in seconds (no seat turns), and the build loop picks up the new milestones under the same check_phase/run_love2d gates.

---

How it works — pipeline + check_phase

One checkpointed LangGraph state machine (the pipeline) gates every advance on the gateway's deterministic check_phase. The seat's job is to run the right tools; the pipeline verifies they ran and passed — no model ever judges prose (the old review graph was retired in M3).

flowchart LR
    S([brief]) --> P0[phase 0<br/>repo] --> P1[1 concept] --> P2[2 storyboards]
    P2 --> P3[3 asset specs] --> P4[4 asset QA] --> P5[5 devplan]
    P5 --> PD[parse_devplan] --> MS[milestone loop<br/>code · test · run_love2d]
    MS --> FV[final_verify<br/>interactive playtest] --> D([done])
    P1 -. "check_phase red → retry" .-> P1
    MS -. retry .-> MS
    P1 -. attempts exhausted .-> B[blocked<br/>resume: retry / skip / abort]

One milestone turn, end to end:

sequenceDiagram
    participant P as pipeline (laptop)
    participant G as gateway (check_phase + ledger)
    participant S as seat (opencode · Qwen3.6-27B)
    participant Z as zoo (comfyui/audio/playtest)

    P->>G: check_phase(milestone) — already green?<br/>(commit-keyed lookup)
    G-->>P: red + findings
    P->>S: milestone prompt (+ last rejection notes)
    S->>Z: generate_image / generate_music / run_love2d …
    Z-->>S: /files/… artifacts + screenshots
    S->>G: critique_image (writes ledger row by URL)
    S->>S: code · test · commit · push
    S-->>P: turn ends (or deadline abort)
    P->>G: check_phase(milestone) again
    G-->>P: green → advance · red → retry with findings

A stall watchdog aborts a wedged turn; the retry cap flips the thread to a resumable blocked state.

check_phase is the single gate (M3)

The gateway runs every deterministic check for the phase server-side and returns {green, findings}. The pipeline re-runs it to advance — it trusts git + the filesystem + the gateway's own records, never the seat's word. A per-game tool ledger keyed by commit SHA makes a passing run_love2d record only count for the exact HEAD it ran on, killing the "passed earlier, ship it" fabrication class. The ledger is gateway-side SQLite (write-once by the tools, read by check_phase); critique_image/analyze_music rows key on asset URL, run_love2d rows key on commit.

flowchart LR
    C([check_phase]) --> GIT[git: HEAD == origin/main<br/>after fetch]
    GIT --> DOC[docs valid<br/>vs scaffolded skeleton]
    DOC --> ASS[assets ↔ docs/03 1:1<br/>every Result has /files/ URL]
    ASS --> TESTS[tests green +<br/>run_love2d artifact]
    TESTS --> CRIT[critique_image PASS record<br/>per asset + per labeled screenshot]
    CRIT -->|all green| OK([green])
    CRIT -->|any red| RED([findings → retry])

Session lifecycle (M5)

The LangGraph pipeline owns session lifecycle: fresh opencode session per phase (and per milestone in phase 6), opened and retired by the graph with committed repo state + the ledger as the only memory between them. Retries stay warm within a phase; nothing warm survives a phase boundary. Seat t/s degrades sharply as context deepens, so the previous "two long sessions" design spent most of its life in the slow tail.

Seat vision

Everything the seat sees — critique_image, playtest-screenshot review, describe — routes to the seat's own vision tower via the gateway (ORCH_VISION_URL=http://brain:8090). A 122B fallback (topped the filtered vision board) stays on disk; flip the URL if self-critique proves too soft.

Batch assets (M4)

At a phase/milestone boundary the studio graph hits a batch_assets node: the seat submits manifests via submit_batch and ends its turn; a small conductor pauses the brain, swaps the generator onto the borrowed GPUs, fires the queue, then restores the brain. Pipeline resumes on batch_status=done. The CPU zoo stays up throughout for realtime drafts between batch windows.

---

Build, deploy, operate

Everything is driven from the laptop; Docker commands run on Trogdor over SSH. The stack is a Portainer git stack: Portainer pulls the compose file from Forgejo, so commit + push before make redeploy — a pushed image alone changes nothing, and the webhook only recreates containers when the git stack changed.

make build       # build the custom images on Trogdor (native amd64, over SSH)
make push        # push them to the Forgejo registry (serial — parallel races Forgejo)
make deploy      # build → push → webhook redeploy (use after image/code changes)
make redeploy    # config-only change: git push first, then poke the webhook
make warm        # after a seat redeploy: pays the one-time ~2 min CUDA init
make health      # gateway + all downstreams
make smoke       # every endpoint + model state, one screen
make ps / logs   # container status / tails (S=brain|comfyui|audio|playtest|mcp)
make test        # 232 laptop tests — pipeline, check_phase, ledger, gateway
make numa        # print Trogdor's NUMA topology + suggested cpusets

Custom images (comfyui, audio, mcp, playtest) build from docker-compose.build.yml; brain and qdrant are stock upstream images. Rollbacks are single env vars in the Portainer stack (or .env) — the compose file's brain comment documents the exact knobs (BRAIN_GGUF, BRAIN_IMAGE_MIN_TOKENS, mmproj variant).

Set up OpenCode

The studio has no built-in agent loop — OpenCode points at the seat and the tool gateway:

• Model (seat): http://trogdor:8090/v1 (OpenAI-compatible)
• Tools (zoo): http://trogdor:8080/mcp/ (trailing slash required)

Game projects live in their own folders, so make opencode-setup symlinks the studio config into ~/.config/opencode/ — every session gets AGENTS.md, the studio agent, plugins, and commands; a git pull here updates them all, and retired files are pruned (a dangling plugin symlink makes OpenCode print a load error on every launch). The installer never touches your global opencode.json[c] — it verifies the trogdor blocks and prints what to paste on drift.

make opencode-setup    # symlinks + prune + verify wiring
opencode               # in ANY folder; /mcp lists the zoo tools

Prompting the seat

The seat calls the zoo's tools directly and judges its own images. Quick reference:

Goal	How
New game	/new-game <brief> (manual) or make studio-run (autonomous) — the six-phase process end to end.
Resume	In the game dir: opencode, or make studio-resume DIR=…. The committed docs/ is the handoff state.
Playtest	/playtest [seconds] — commits + pushes, run_love2d(git_url=…), critiques each screenshot against the storyboard.
Image (draft)	generate_image(transparent=true, sprite_size=256) on the live CPU zoo (klein/Z-Image Turbo) — fast iteration.
Image (final)	submit_batch(image, [manifest]) at a milestone boundary — conductor swaps to FLUX.2 [dev] on ALL 4 GPUs, ships finals.
Spritesheet	"4-frame walk-cycle of a knight walking right." → generate_spritesheet(frames=4) — alpha strip + per-frame PNGs + Love2D quads in one pass. Never generate frames separately.
Animation	animate_sprite enqueues into the animate profile (Wan 2.2 image→video→alpha-matted sheet).
Music / SFX	generate_music / generate_sfx (async — poll check_job), then analyze_music (CLAP intent score >0.4 strong).
Video	generate_video (LTX-Video via ComfyUI, async) — used for intros/cutscenes; server-side .ogv for LÖVE playback.
Speech / transcription	generate_tts (Chatterbox); transcribe_audio (faster-whisper, speech only — detect_speech first on mixed clips).
Library asset	search_local_assets("wooden table") (semantic) · find_similar_assets(url) (reverse-image). Results carry a license — credit CC-BY.

Briefing rules (generators have no memory — each prompt stands alone): be specific (subject, style, palette, perspective, use); search the library first; slow generators are async {job_id} jobs (poll check_job, never resubmit); close the loop with verify:true or critique_image.

The full operating manual is AGENTS.md, auto-loaded into every session.

Services

One compose file, six services. The brain (seat) takes all 4 GPUs during turns; comfyui lives on GPU2 and audio on GPU3 for the rest of the time (no more model-reload thrash between image and audio phases). Batch swaps the generator onto the borrowed GPUs with the brain paused. The gateway (mcp) is the only service the seat talks to directly.

Service	Container · port · compute	Model	Capability
brain (studio seat)	trogdor-coder · 8090 · all 4 GPUs	Qwen3.6-27B dense + mmproj-BF16 vision · 262K ctx · q8 KV · ~38 GB VRAM	Designs, generates, codes; ~35.6 t/s decode / 506 pp; serves all vision (critique_image / describe / playtest review).
mcp (gateway)	trogdor-mcp · 8080 · CPU	n/a (orchestrator)	MCP tools + REST + /files + /library; routes every service; runs async jobs + the batch queue; hosts check_phase + the per-game tool ledger.
comfyui	gen-comfyui · 8188 · GPU2	FLUX.2 klein 9B (GGUF, live/draft) · Z-Image Turbo (batch profile) · Wan 2.2 (animate) · HunyuanVideo (video) · BiRefNet matting	Image, spritesheet, video. Batch profiles use ALL 4 GPUs during batch windows.
audio	gen-audio · 8092 · GPU3	ACE-Step 1.5 · Stable Audio Open · AudioGen · Chatterbox · faster-whisper · Silero VAD · CLAP + librosa	Music, SFX, TTS, ASR, VAD, analysis. Batch: 3 parallel seats on GPU0–2.
playtest	trogdor-playtest · 8093 · CPU	LÖVE under Xvfb + ffmpeg	Headless run + screenshots; /love2d today (one endpoint per engine).
qdrant (+ library mount)	trogdor-qdrant · 6333 · CPU	n/a	CLIP dense + BM25 sparse over the ~90k-file CC0 library; the gateway embeds via fastembed.

Testing each service by hand

Every generated asset lands under /mnt/output and is served at http://trogdor:8080/files/… — the JSON response carries the URL; open it in a browser to preview.

# Gateway + everything downstream:
curl -s http://trogdor:8080/health | python3 -m json.tool
python3 scripts/smoke.py                      # the full ✅/❌ tree

# Seat (raw completion + throughput timings):
curl -s http://trogdor:8090/completion -H 'content-type: application/json' \
  -d '{"prompt":"-- lua: shuffle a deck\n","n_predict":64}' | python3 -m json.tool

# Image → preview URL:
curl -s http://trogdor:8080/assets/image -H 'content-type: application/json' \
  -d '{"prompt":"pixel art treasure chest, closed, side view","transparent":true,"sprite_size":128}'
#   → {"url": "http://trogdor:8080/files/…png", …}   ← open in a browser

# Music (60s loop) and SFX:
curl -s http://trogdor:8080/assets/music -H 'content-type: application/json' \
  -d '{"prompt":"upbeat chiptune town theme, loopable","seconds":60}'
curl -s http://trogdor:8080/assets/sfx -H 'content-type: application/json' \
  -d '{"prompt":"coin pickup, bright, short"}'

# Speech:
curl -s http://trogdor:8080/assets/tts -H 'content-type: application/json' \
  -d '{"text":"Welcome to the dungeon."}'

# Vision (the seat's own eyes, via the gateway):
curl -s http://trogdor:8080/describe -H 'content-type: application/json' \
  -d '{"url":"http://trogdor:8080/files/<something>.png"}'

# Headless playtest of any pushed game repo:
curl -s http://trogdor:8093/love2d -H 'content-type: application/json' \
  -d '{"git_url":"https://forgejo.underthere.xyz/cmoriarty/<game>.git","seconds":20}'

Video generation is exposed as the generate_video MCP tool (async job) — easiest driven from an OpenCode session; the result is an /files/… URL like everything else.

REST surface (what exists besides the MCP tools)

Method	Path	Purpose
GET	/health	Liveness + downstream reachability
GET	/files/{path} · /library/{path}	Fetch/preview a generated / CC0 asset
POST	/assets/image | /music | /sfx | /tts	One-shot generation → JSON record with /files URL
POST	/assets/upload	Multipart file → /files URL
POST	/describe · /critique	REST twins of read_image / critique_image
POST	/transcribe	ASR (faster-whisper)
POST	/compose · /extract	Image composition · doc table extraction
POST	/check-phase	The deterministic phase gate the pipeline calls
POST/GET/PUT	/studio/session · /studio/active-game	Pipeline ↔ gateway wiring (session registry, active game)

Hardware (Trogdor)

Component	Specification
Server	Dell T640
CPU	2× Intel Xeon Silver 4214 — 24 cores / 48 threads, 2 NUMA nodes
GPU	4× NVIDIA Quadro RTX 5000 — 16 GB each (Turing; GPU 0+1 NVLink)
Memory	384 GB DDR4-2400 — 192 GB per NUMA node
Storage	4× 1 TB SAS SSD, RAID 10 (~2 TB usable)

The dense 27B seat fits all 4 GPUs in VRAM (~38 GB). Batch swaps may overload the host during MoE offload (e.g. FLUX.2 [dev] batch profile); kernel.numa_balancing=0 is load-bearing then (+47% decode during brain-down windows).