nzxt-qwen36-35b-a3b

Qwen3.6-35B-A3B (unsloth UD-IQ4_XS, 16.5 GiB) + vision mmproj on stock llama.cpp, on nzxt (RTX A4500 20 GB). Portainer git stack, endpoint 43, served at http://nzxt:8000/v1 as model id qwen3.6-35b-a3b.

Purpose: a parallel lane for opencode's explore scout and a vision-reviewer subagent, so background work stops taxing the trogdor primary. Nothing quality-bearing runs here.

Deploy

# model + template (done 2026-09-04)
ssh nzxt 'hf download unsloth/Qwen3.6-35B-A3B-GGUF \
  --include Qwen3.6-35B-A3B-UD-IQ4_XS.gguf --include mmproj-F16.gguf \
  --local-dir /home/models/Qwen3.6-35B-A3B-GGUF
 curl -sL -o /home/models/Qwen3.6-35B-A3B-GGUF/chat_template_fixed.jinja \
  https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/resolve/main/chat_template.jinja'
# Portainer: git stack on this repo, endpoint 43, compose docker-compose.yml

Verification protocol

1. Boot + VRAM headroom (raise -c if > 1 GiB free).
2. API probe: streaming, reasoning_content, tool_calls, vision, system message ordering (froggeric template has no order trap — confirm).
3. Speed: short/20k decode, cold prefill, cached TTFT, concurrency 2.
4. Real opencode run --agent explore on a known repo: 5 "where is X / who calls Y" questions scored against ground truth.

Results are appended below as they land.

Results (2026-09-04)

Boot: healthy in ~20s (weights stream from /home at cold boot); at -c 131072 q8_0 KV VRAM is 19.7 of 20.5 GiB (0.8 GiB headroom — stop here).

API probe (all pass): system message not-first (froggeric template has no ordering trap), multiple system messages, tool_calls JSON, stray reasoning_effort field ignored, vision via mmproj. Thinking arrives in reasoning_content (--reasoning-format deepseek).

Speed (single stream unless noted, 64k-config boot; 131k identical):

metric	nzxt Qwen3.6-35B-A3B	trogdor vLLM 27B	trogdor Flash-Next
decode, short ctx	110 t/s	56	28.6
decode @20k	97 t/s	51	25.0
cold 21k prefill	5.8 s	9.8 s	45 s
cached-turn TTFT	0.13–0.34 s	0.34 s	2.3 s
conc-2 per-stream / aggregate	47 + 78 / 125	—	15 + 17 / 32
prefix-cache hit, multi-turn	99.6%	—	99.8%

A 3B-active MoE on Ampere out-runs both trogdor arms for the scout role. Quality is the lower tier (AA 26) by design — nothing quality-bearing runs here.

Explore quality gate (2026-09-04) — 5/5. Driven the real way: build primary on trogdor → task tool → @explore on nzxt, answers pulled from the subagent sessions in opencode's store, scored against grep ground truth in qwen38-test:

question	expected	explore (nzxt)
where is computeFeedback	src/rules.js:41	✓
where is pickAnswer	src/rules.js:12	✓
callers of hintWord	hints.js, scene.js, test/hints.test.js	✓ (+2 openspec docs mentioning the name)
MAX_GUESSES / WORD_LENGTH	rules.js:5 = 6, rules.js:6 = 5	✓
merge fn in wordlists.js	mergeDefinitions :27	✓

Cost per task on nzxt: 573–4,728 input tokens, 106–228 output — seconds. The ~4 min wall clock per question in the harness was the flash-next primary prefilling its 24k build prompt and thinking around the delegation, not the scout. opencode run --agent <subagent-mode agent> hangs silently (never creates a session) — always drive subagents through a primary with an @mention.

Vision-reviewer gate (2026-09-04) — PASS. build → @vision-reviewer on a synthetic 640×400 UI screenshot with a stated expectation. Report: correct verdict, counted all six text-like bars and noticed they shorten progressively, estimated the header color (#2255C4 vs actual #1E50C8), placed every element correctly. 10,773 input / 425 output tokens on nzxt; the primary never loaded the image. First run died on opencode's external_directory permission (auto-rejected headless) — fixed with "external_directory": "allow" on this read-only agent.

Reader gate (2026-09-05) — PASS. New read-only reader subagent on this lane (research/docs/large files/logs). Driven build → @reader:

• File digest of trog/experiments/nzxt-a4500/RESEARCH.md: decode 32.4 / 31.5 t/s, prefill 1.0k t/s, KV 13 KiB/token — exact ground truth, with line citations. 61 s end-to-end, one read call.
• Web lookup, latest llama.cpp release: v0.4.0 (2026-09-04), and it correctly noted b10816 is a pre-release published a minute later — a more careful answer than the naive "newest tag". Two webfetch calls. Both ran on nzxt/qwen3.6-35b-a3b; the 35k-token fetched page stayed in this lane's pool and never touched the primary's context.