cmoriarty/trog

Fork 0

03 · MCP server: deterministic verification and git truth #5

New issue

Closed

opened 2026-07-23 00:07:08 -04:00 by cmoriarty · 2 comments

cmoriarty commented

2026-07-23 00:07:08 -04:00

Owner

Verification lives in tools, not prompts.

Architecture (decided): verification and batch logic is a plain Python library; LangGraph nodes import it directly (same process, no HTTP hop). A thin, pure MCP server wraps that library for the brain — the LLM is the only MCP consumer. No parallel REST surface like agentic-game-dev's gateway (its ~28 REST routes partly exist because the orchestrator predates the graph).

Core capabilities:

check_phase — deterministic checks (files exist, tests pass, assets present, critiques recorded), keyed to the exact commit it ran on
Commit-keyed ledger of verification results — a new commit invalidates old passes; the graph's conditional edges consult the same ledger
One Forgejo repo per game — git is truth

Reuse policy: agentic-game-dev's orchestrator/orchestrator/main.py exposes 26 MCP tools + ~28 REST routes. Audit every one before porting; expect to drop many. Each tool carried over needs a reason tied to a v1 ticket — nothing comes along by default (e.g. CC0 asset search, qdrant indexing, studio session routes are likely cuts).

Reference: /Users/cmoriarty/repos/agentic-game-dev/orchestrator/orchestrator/ — check_phase.py, ledger.py, batch.py, main.py.

Verification lives in tools, not prompts. **Architecture (decided):** verification and batch logic is a plain Python library; LangGraph nodes import it directly (same process, no HTTP hop). A thin, pure **MCP server** wraps that library for the brain — the LLM is the only MCP consumer. No parallel REST surface like agentic-game-dev's gateway (its ~28 REST routes partly exist because the orchestrator predates the graph). Core capabilities: - `check_phase` — deterministic checks (files exist, tests pass, assets present, critiques recorded), keyed to the exact commit it ran on - Commit-keyed ledger of verification results — a new commit invalidates old passes; the graph's conditional edges consult the same ledger - One Forgejo repo per game — git is truth **Reuse policy:** agentic-game-dev's `orchestrator/orchestrator/main.py` exposes 26 MCP tools + ~28 REST routes. **Audit every one before porting; expect to drop many.** Each tool carried over needs a reason tied to a v1 ticket — nothing comes along by default (e.g. CC0 asset search, qdrant indexing, studio session routes are likely cuts). Reference: `/Users/cmoriarty/repos/agentic-game-dev/orchestrator/orchestrator/` — `check_phase.py`, `ledger.py`, `batch.py`, `main.py`.

cmoriarty added this to the v1 - single user autonomous milestone

2026-07-23 00:07:08 -04:00

cmoriarty changed title from ~~Gateway: deterministic verification and git truth~~ to 03 · Gateway: deterministic verification and git truth

2026-07-23 00:10:05 -04:00

cmoriarty changed title from ~~03 · Gateway: deterministic verification and git truth~~ to 03 · MCP server: deterministic verification and git truth

2026-07-23 00:22:33 -04:00

cmoriarty referenced this issue from a commit

2026-07-23 00:39:37 -04:00

docs(readme): rename gateway to mcp server

cmoriarty referenced this issue from a commit

2026-07-23 11:58:50 -04:00

feat: trog_lib — deterministic verification and git truth

cmoriarty commented

2026-07-23 12:47:00 -04:00

Author

Owner

Done — verification is deterministic, git is truth

What was done (main commits: 53134e0, 6516ba3)

server/trog_lib/ — the plain library, exactly as this ticket decided: graph nodes import it in-process, no HTTP hop, no REST surface.
- forgejo.py — per-game repos under the trog-games org via the Forgejo API (no git binary in the container): create repo, commit file, read file at any ref
- ledger.py — commit-keyed, append-only verification ledger in the stack's Postgres. Producers write rows as side effects; consumers do record lookups. A verdict is bound to the exact commit it judged — edit a file and every old record is stale by construction (design inherited from agentic-game-dev's ledger, sqlite → Postgres)
- check_phase.py — deterministic checks at an exact commit: doc committed, doc substantial, no scaffold {placeholders} left, passing gate critique on the ledger for that same commit
trog_lib/mcp_server.py → new compose service trog-mcp (:8100, streamable-http, same image as aegra, different command). Two tools: check_phase, ledger_recent. The LLM is the only intended MCP consumer.
Production graph — new setup_repo entry node creates the game repo + commits the brief; every phase commits its doc; gates record the brain-critique on the ledger and then route on check_phase facts. The critic advises, the ledger decides.
docs/mcp-audit.md — full disposition of agentic-game-dev's 26 MCP tools + 28 REST routes: 1 ported now (check_phase, reshaped), ~14 deferred to ticket 05 with reasons, the rest cut (CC0 search, web tools, Löve runner, speech, card compositing, studio-session REST).
Resilience fix found by a live failure: a Portainer image pull recreated the brain mid-run and the first node died on llama.cpp's 503 "Loading model". setup_repo now health-gates the run's entry; the LLM client retries transient 5xx.

Live evidence (brief: herd fireflies into a jar before dawn):

The fun gate rejected two GDDs before passing the third — the backward edge fired for real, and the repo + ledger recorded all of it:

git log (trog-games/a-tiny-browser-game-2607231604):
  2c381bc ideation: brief          eb57f67 preproduction: gdd  (v3)
  80a2e29 ideation: concept        57d8f07 vertical_slice: slice
  4f7910c preproduction: gdd (v1)  a819499 production: build
  7ddb246 preproduction: gdd (v2)  7f35618 alpha … 1a3ad49 ship

ledger:
  gate_fun fail @4f7910c → check_phase fail @4f7910c   (back)
  gate_fun fail @7ddb246 → check_phase fail @7ddb246   (back)
  gate_fun pass @eb57f67 → check_phase pass @eb57f67   (advance)
  gate_bar pass @57d8f07 → check_phase pass @57d8f07   (advance)

Also live-verified: check_phase FAILS before a critique is recorded and for stale commits; the MCP endpoint serves both tools to a real streamable-http client cross-container.

How to test

make test        # 15 tests, no network
scripts/produce-test.sh <host> "an idea"   # ~25-45 min; then browse the repo it names
ssh <gpu-host> "docker exec trog-postgres psql -U aegra -d aegra -c 'select tool, verdict, left(commit_sha,7), target from trog_ledger order by id desc limit 10'"

Notes

FORGEJO_TOKEN (write:repository + write:organization) lives only in the Portainer stack env.
Gate loops cost real time: the firefly run took 41 min because the fun gate demanded two rewrites. Working as designed — but ticket 10 should budget for it.
Portainer redeploys should keep pullImage: false; a surprise upstream pull is what killed run one.

## Done — verification is deterministic, git is truth **What was done** (main commits: 53134e0, 6516ba3) - **`server/trog_lib/`** — the plain library, exactly as this ticket decided: graph nodes import it in-process, no HTTP hop, no REST surface. - `forgejo.py` — per-game repos under the **trog-games** org via the Forgejo API (no git binary in the container): create repo, commit file, read file at any ref - `ledger.py` — commit-keyed, append-only verification ledger in the stack's Postgres. Producers write rows as side effects; consumers do record lookups. A verdict is bound to the exact commit it judged — edit a file and every old record is stale by construction (design inherited from agentic-game-dev's ledger, sqlite → Postgres) - `check_phase.py` — deterministic checks at an exact commit: doc committed, doc substantial, no scaffold `{placeholders}` left, passing gate critique on the ledger *for that same commit* - **`trog_lib/mcp_server.py`** → new compose service **trog-mcp** (:8100, streamable-http, same image as aegra, different command). Two tools: `check_phase`, `ledger_recent`. The LLM is the only intended MCP consumer. - **Production graph** — new `setup_repo` entry node creates the game repo + commits the brief; every phase commits its doc; gates record the brain-critique on the ledger and then route on `check_phase` facts. The critic advises, the ledger decides. - **`docs/mcp-audit.md`** — full disposition of agentic-game-dev's 26 MCP tools + 28 REST routes: 1 ported now (check_phase, reshaped), ~14 deferred to ticket 05 with reasons, the rest cut (CC0 search, web tools, Löve runner, speech, card compositing, studio-session REST). - Resilience fix found by a live failure: a Portainer image pull recreated the brain mid-run and the first node died on llama.cpp's 503 "Loading model". `setup_repo` now health-gates the run's entry; the LLM client retries transient 5xx. **Live evidence** (brief: *herd fireflies into a jar before dawn*): The fun gate rejected two GDDs before passing the third — the backward edge fired for real, and the repo + ledger recorded all of it: ``` git log (trog-games/a-tiny-browser-game-2607231604): 2c381bc ideation: brief eb57f67 preproduction: gdd (v3) 80a2e29 ideation: concept 57d8f07 vertical_slice: slice 4f7910c preproduction: gdd (v1) a819499 production: build 7ddb246 preproduction: gdd (v2) 7f35618 alpha … 1a3ad49 ship ledger: gate_fun fail @4f7910c → check_phase fail @4f7910c (back) gate_fun fail @7ddb246 → check_phase fail @7ddb246 (back) gate_fun pass @eb57f67 → check_phase pass @eb57f67 (advance) gate_bar pass @57d8f07 → check_phase pass @57d8f07 (advance) ``` Also live-verified: check_phase FAILS before a critique is recorded and for stale commits; the MCP endpoint serves both tools to a real streamable-http client cross-container. **How to test** ```bash make test # 15 tests, no network scripts/produce-test.sh <host> "an idea" # ~25-45 min; then browse the repo it names ssh <gpu-host> "docker exec trog-postgres psql -U aegra -d aegra -c 'select tool, verdict, left(commit_sha,7), target from trog_ledger order by id desc limit 10'" ``` **Notes** - `FORGEJO_TOKEN` (write:repository + write:organization) lives only in the Portainer stack env. - Gate loops cost real time: the firefly run took 41 min because the fun gate demanded two rewrites. Working as designed — but ticket 10 should budget for it. - Portainer redeploys should keep `pullImage: false`; a surprise upstream pull is what killed run one.

cmoriarty commented

2026-07-23 16:29:38 -04:00

Author

Owner

Review complete — closing

Code and the generated game repo reviewed and approved.

Wrap-up:

Smoke-test artifacts cleaned: trog-games/a-tiny-browser-game-2607231601 (aborted run one) and …2607231604 (the full firefly run) both deleted — the ledger design means their evidence lives in this ticket's comment, not in kept repos
Post-review docstring trim landed as d030d53
CI now guards this work going forward: .forgejo/workflows/test.yml runs the 15-test suite on every push/PR via the new instance runner (strongbad-runner), and check-updates.yml reports dependency drift daily — both badged in the README

Verification remains deterministic, git remains truth. On to ticket 05/10.

## Review complete — closing Code and the generated game repo reviewed and approved. **Wrap-up:** - Smoke-test artifacts cleaned: `trog-games/a-tiny-browser-game-2607231601` (aborted run one) and `…2607231604` (the full firefly run) both deleted — the ledger design means their evidence lives in this ticket's comment, not in kept repos - Post-review docstring trim landed as d030d53 - CI now guards this work going forward: `.forgejo/workflows/test.yml` runs the 15-test suite on every push/PR via the new instance runner (strongbad-runner), and `check-updates.yml` reports dependency drift daily — both badged in the README Verification remains deterministic, git remains truth. On to ticket 05/10.

cmoriarty closed this issue

2026-07-23 16:29:39 -04:00

cmoriarty referenced this issue

2026-07-23 17:56:30 -04:00

10 · E2E walking skeleton: brief → finished game, fully autonomous #11