03.1 · Brain toolbelt: survey existing skills, tools, and MCP servers #12

New issue

Closed

opened 2026-07-23 14:39:43 -04:00 by cmoriarty · 1 comment

cmoriarty commented

2026-07-23 14:39:43 -04:00

Owner

Before building more tools ourselves, survey what already exists that could make the studio brain more capable and reliable. Precedent: agentic-game-dev's web_search/web_read (searxng + server-side extraction) measurably reduced failed web reads versus letting the model fetch raw pages — cheap wins like that likely exist elsewhere.

To investigate:

Code agent choice — OpenCode vs Qwen-native alternatives — OpenCode is the incumbent (and well-liked); question is whether a Qwen-family agent (QwenCode, etc.) buys tighter integration with the llama.cpp Qwen3.6 brain — chat-template/tool-call fidelity, parallel tool calls, thinking-mode handling, parser-level support. Bar to switch: must beat OpenCode on measurable reliability with the local brain, not vibes. Research, decide, document either way.
Web research tools — revive web_search/web_read? Requires deciding whether searxng rejoins the compose stack, or whether an existing MCP web-fetch server does the job (server-side readability extraction is the important part — raw HTML wastes context and fails often).
Phaser's own agent support — Phaser 4 ships AI agent skills in-repo (noted in our README). If the framework maintains its own LLM-facing docs/skills, that beats anything we'd write. Check Three.js/Rapier ecosystems for equivalents.
Library-docs MCPs (e.g. Context7-style) — up-to-date API docs on demand instead of the brain hallucinating Phaser APIs from training data. Evaluate token cost vs benefit.
Playwright/browser MCP servers — ticket 04 (playtest) may be mostly buildable from an existing browser-automation MCP rather than a custom harness. Worth checking before writing one.
Reference MCP servers (filesystem, git, memory) — what's mature enough to trust; what overlaps with trog_lib's forgejo module and should NOT be duplicated.
Skill formats — Anthropic's SKILL.md / OpenCode skills / similar packaged-workflow formats: could phase prompts + scaffolds ship as skills the brain loads, instead of hardcoded prompt strings in production.py?
MCP registries (official registry, Smithery, etc.) — do one sweep for game-dev-adjacent servers: asset pipelines, aseprite/tilemap tooling, audio.
Vision/critique tooling — anything existing for screenshot review that beats rolling our own for ticket 04/05 critics.

Evaluation bar for anything adopted: open source (MIT/Apache), self-hosted, no phone-home, works over MCP with a local llama.cpp brain (tool-calling via chat template), and the tool-schema token overhead must be worth it — a 27B brain with 262K ctx still pays for every tool description on every turn.

Deliverable: a short audit doc (docs/toolbelt-audit.md) with keep/skip/later per candidate + wiring plan for the keepers (served via trog-mcp vs sidecar MCP services), and follow-up issues for anything that lands in a later ticket's scope.

Before building more tools ourselves, survey what already exists that could make the studio brain more capable and reliable. Precedent: agentic-game-dev's `web_search`/`web_read` (searxng + server-side extraction) measurably reduced failed web reads versus letting the model fetch raw pages — cheap wins like that likely exist elsewhere. **To investigate:** - **Code agent choice — OpenCode vs Qwen-native alternatives** — OpenCode is the incumbent (and well-liked); question is whether a Qwen-family agent (QwenCode, etc.) buys tighter integration with the llama.cpp Qwen3.6 brain — chat-template/tool-call fidelity, parallel tool calls, thinking-mode handling, parser-level support. Bar to switch: must beat OpenCode on measurable reliability with the local brain, not vibes. Research, decide, document either way. - **Web research tools** — revive `web_search`/`web_read`? Requires deciding whether searxng rejoins the compose stack, or whether an existing MCP web-fetch server does the job (server-side readability extraction is the important part — raw HTML wastes context and fails often). - **Phaser's own agent support** — Phaser 4 ships AI agent skills in-repo (noted in our README). If the framework maintains its own LLM-facing docs/skills, that beats anything we'd write. Check Three.js/Rapier ecosystems for equivalents. - **Library-docs MCPs** (e.g. Context7-style) — up-to-date API docs on demand instead of the brain hallucinating Phaser APIs from training data. Evaluate token cost vs benefit. - **Playwright/browser MCP servers** — ticket 04 (playtest) may be mostly buildable from an existing browser-automation MCP rather than a custom harness. Worth checking before writing one. - **Reference MCP servers** (filesystem, git, memory) — what's mature enough to trust; what overlaps with trog_lib's forgejo module and should NOT be duplicated. - **Skill formats** — Anthropic's SKILL.md / OpenCode skills / similar packaged-workflow formats: could phase prompts + scaffolds ship as skills the brain loads, instead of hardcoded prompt strings in `production.py`? - **MCP registries** (official registry, Smithery, etc.) — do one sweep for game-dev-adjacent servers: asset pipelines, aseprite/tilemap tooling, audio. - **Vision/critique tooling** — anything existing for screenshot review that beats rolling our own for ticket 04/05 critics. **Evaluation bar for anything adopted:** open source (MIT/Apache), self-hosted, no phone-home, works over MCP with a local llama.cpp brain (tool-calling via chat template), and the tool-schema token overhead must be worth it — a 27B brain with 262K ctx still pays for every tool description on every turn. **Deliverable:** a short audit doc (`docs/toolbelt-audit.md`) with keep/skip/later per candidate + wiring plan for the keepers (served via trog-mcp vs sidecar MCP services), and follow-up issues for anything that lands in a later ticket's scope.

cmoriarty added this to the v1 - single user autonomous milestone

2026-07-23 14:39:43 -04:00

cmoriarty changed title from ~~3.1 · Brain toolbelt: survey existing skills, tools, and MCP servers~~ to 03.1 · Brain toolbelt: survey existing skills, tools, and MCP servers

2026-07-23 14:39:53 -04:00

cmoriarty commented

2026-07-23 16:38:04 -04:00

Author

Owner

Done — audit complete, OpenCode stays

Deliverable: docs/toolbelt-audit.md (commit 548618c) — keep/later/skip per candidate + wiring plan.

The code-agent decision (the added task): keep OpenCode. Key finding: with llama.cpp's --jinja chat template doing Qwen tool-call parsing server-side, both OpenCode and Qwen Code see the identical OpenAI tools API — there is no client-side integration edge left for Qwen Code to claim. Its free OAuth tier also died 2026-04-15, so no cost angle either. OpenCode's native Agent Skills (SKILL.md) support pairs directly with Phaser 4's in-repo skills. Revisit trigger documented: persistent tool-call failure rate in ticket 10 → cheap A/B against Qwen Code (same endpoint config).

Keeps: Phaser 4's in-repo skills/ (~28 files, framework-maintained, zero token cost — ticket 10); Playwright MCP for playtest (ticket 04, canvas caveat documented: screenshot path + brain vision, not a11y snapshots); searxng + webfetch-mcp for web research (new issue below); SKILL.md as the format for agent-facing phase prompts.

Skips: reference MCP servers (filesystem/git duplicate trog_lib.forgejo + OpenCode's own tools), SaaS asset generators (fail the self-host bar), external vision-critique tooling (nothing mature; brain has vision + ported critics).

Laters: library-docs MCP (only if Phaser skills don't stop API hallucination — arabold/docs-mcp-server is the candidate), aseprite MCPs (logged for ticket 05), Three.js/Rapier llms.txt (when a 3D ticket exists).

Follow-ups filed/noted: new issue #13 (web research wiring); pointers left in the audit for tickets 04/05/10.

Token-budget outcome: the brain's standing toolbelt stays at 4 tools (2 trog_lib + 2 web). Playwright's ~20 tools mount only inside the playtest service.

## Done — audit complete, OpenCode stays **Deliverable:** [docs/toolbelt-audit.md](https://forgejo.underthere.xyz/cmoriarty/trog/src/branch/main/docs/toolbelt-audit.md) (commit 548618c) — keep/later/skip per candidate + wiring plan. **The code-agent decision (the added task):** **keep OpenCode.** Key finding: with llama.cpp's `--jinja` chat template doing Qwen tool-call parsing server-side, both OpenCode and Qwen Code see the identical OpenAI tools API — there is no client-side integration edge left for Qwen Code to claim. Its free OAuth tier also died 2026-04-15, so no cost angle either. OpenCode's native Agent Skills (SKILL.md) support pairs directly with Phaser 4's in-repo skills. Revisit trigger documented: persistent tool-call failure rate in ticket 10 → cheap A/B against Qwen Code (same endpoint config). **Keeps:** Phaser 4's in-repo `skills/` (~28 files, framework-maintained, zero token cost — ticket 10); Playwright MCP for playtest (ticket 04, canvas caveat documented: screenshot path + brain vision, not a11y snapshots); searxng + webfetch-mcp for web research (new issue below); SKILL.md as the format for agent-facing phase prompts. **Skips:** reference MCP servers (filesystem/git duplicate trog_lib.forgejo + OpenCode's own tools), SaaS asset generators (fail the self-host bar), external vision-critique tooling (nothing mature; brain has vision + ported critics). **Laters:** library-docs MCP (only if Phaser skills don't stop API hallucination — arabold/docs-mcp-server is the candidate), aseprite MCPs (logged for ticket 05), Three.js/Rapier llms.txt (when a 3D ticket exists). **Follow-ups filed/noted:** new issue #13 (web research wiring); pointers left in the audit for tickets 04/05/10. Token-budget outcome: the brain's standing toolbelt stays at 4 tools (2 trog_lib + 2 web). Playwright's ~20 tools mount only inside the playtest service.

cmoriarty closed this issue

2026-07-23 17:09:13 -04:00