cmoriarty/trog
1
0
Fork 0
Code Issues 11 Pull requests Projects 1 Releases Packages Wiki Activity Actions

07 · Studio: live agent feed — chat, todos, prompt insertion #8

New issue
Closed
opened 2026-07-23 00:07:12 -04:00 by cmoriarty · 7 comments
cmoriarty commented 2026-07-23 00:07:12 -04:00
Owner
Copy link

The feed becomes a first-class coding-agent chat — the OpenCode/Claude-Code experience, but rendering trog's own LangGraph runs. The #7 shell already streams tokens via useStream; this ticket makes the stream legible.

Core

  • Rich transcript — streaming markdown with syntax-highlighted code blocks; per-message provenance chip naming the graph node that emitted it (ideation, gate_fun, assets…)
  • Tool-call cards — every tool invocation as a collapsed card: tool name, args summary, live spinner while executing, expand for full args/result. MCP attribution on the card: which server answered (trog-mcp · websearch · seat calls), so you can see what the agent is reaching for at a glance
  • Live todo list — the LLM's working plan rendered as it updates (pending / in-progress ✓ done), OpenCode-style. Needs a graph-side contract: a todos state key the nodes maintain; renders from the same values stream
  • Diff cards — file edits shown as unified diffs with highlighting, collapsed to filename + +/- counts
  • Thinking foldout — reasoning streams render dimmed and collapsible (seeded in #7; wire reasoning_content through llama.cpp's reasoning-format flag)
  • Prompt insertion / steering — the input stays live mid-run; sending interrupts the graph, which checkpoints and folds the guidance in. Stop button = hard interrupt

More OpenCode features worth stealing

  • Run timeline strip — per-node duration chips as the run walks (ideation 34s · preprod 1m12s · …) — doubles as a cost/latency profile
  • Context meter — tokens streamed / context used per run, brain busy time; the "how much runway is left" glance
  • Quote-reply — select any transcript text → quoted into the prompt box
  • Notifications — gate reached, run errored, phase changed: browser notification + pixel chime when the studio is backgrounded
  • Thread picker / replay — jump into any past thread from the map's run list and replay its transcript from checkpoints (joinStream for live ones)

Plumbing notes

  • Everything above rides streams we already consume: messages-tuple (tokens + tool_call_chunks), values (todos, phase), updates (node boundaries → provenance + timeline), custom (seat/MCP progress events if graphs emit them)
  • Feed already remounts per selected graph (top-bar picker) and shares the status snapshot — build on that, don't add pollers

v2 note (unchanged): judging is scoped out to v2 — prompt insertion here is steering, not gate judgment. Build the interrupt path so v2's gate-judgment UI drops in without rework.

The feed becomes a **first-class coding-agent chat** — the OpenCode/Claude-Code experience, but rendering trog's own LangGraph runs. The #7 shell already streams tokens via `useStream`; this ticket makes the stream *legible*. ## Core - **Rich transcript** — streaming markdown with syntax-highlighted code blocks; per-message provenance chip naming the **graph node** that emitted it (`ideation`, `gate_fun`, `assets`…) - **Tool-call cards** — every tool invocation as a collapsed card: tool name, args summary, live spinner while executing, expand for full args/result. **MCP attribution** on the card: which server answered (`trog-mcp` · `websearch` · seat calls), so you can see *what* the agent is reaching for at a glance - **Live todo list** — the LLM's working plan rendered as it updates (pending / in-progress ✓ done), OpenCode-style. Needs a graph-side contract: a `todos` state key the nodes maintain; renders from the same values stream - **Diff cards** — file edits shown as unified diffs with highlighting, collapsed to filename + `+/-` counts - **Thinking foldout** — reasoning streams render dimmed and collapsible (seeded in #7; wire `reasoning_content` through llama.cpp's reasoning-format flag) - **Prompt insertion / steering** — the input stays live mid-run; sending interrupts the graph, which checkpoints and folds the guidance in. Stop button = hard interrupt ## More OpenCode features worth stealing - **Run timeline strip** — per-node duration chips as the run walks (`ideation 34s · preprod 1m12s · …`) — doubles as a cost/latency profile - **Context meter** — tokens streamed / context used per run, brain busy time; the "how much runway is left" glance - **Quote-reply** — select any transcript text → quoted into the prompt box - **Notifications** — gate reached, run errored, phase changed: browser notification + pixel chime when the studio is backgrounded - **Thread picker / replay** — jump into any past thread from the map's run list and replay its transcript from checkpoints (`joinStream` for live ones) ## Plumbing notes - Everything above rides streams we already consume: `messages-tuple` (tokens + `tool_call_chunks`), `values` (todos, phase), `updates` (node boundaries → provenance + timeline), `custom` (seat/MCP progress events if graphs emit them) - Feed already remounts per selected graph (top-bar picker) and shares the status snapshot — build on that, don't add pollers **v2 note (unchanged):** judging is scoped out to v2 — prompt insertion here is *steering*, not gate judgment. Build the interrupt path so v2's gate-judgment UI drops in without rework.
cmoriarty changed title from Studio: live agent feed — chat, todos, prompt insertion to 07 · Studio: live agent feed — chat, todos, prompt insertion 2026-07-23 00:10:08 -04:00
cmoriarty commented 2026-07-25 23:24:45 -04:00
Author
Owner
Copy link

Core implemented (deployed to :2027)

All verified live against a phasewalk run in the browser:

  • Rich transcript — streaming markdown (react-markdown + gfm), highlight.js on the Sweetie-16 palette, diff fences as +/- cards; node provenance chips (parsed from the [node] narration prefix)
  • Tool-call cards — name + MCP attribution chip (trog-mcp / websearch mcp / seat), args summary, beat-locked spinner while pending, expandable args/result; calls paired to ToolMessages by tool_call_id
  • Todo list — contract landed: state.todos = [{text, status: pending|in_progress|done}]; collapsible strip with count + current item. phasewalk maintains it across the walk
  • Thinking foldout, run timeline (per-node second chips from updates events — the fun-gate bounce is visible in the chips), token meter (latest turn usage when the model reports it), quote-into-prompt, autoscroll
  • Steering — input stays live mid-run; sending stops the stream and folds the line into the checkpointed thread as the next turn. Verified: steered at alpha, the follow-up walk kept checkpointed state (fun gate passed first try — the bounce disappeared)

phasewalk exercises all of it GPU-free: todos, gates emitting real check_phase tool_calls + results, markdown narration. Structural test asserts both contracts.

Remaining (next passes)

  • notifications (gate / error / done when backgrounded) + pixel chime
  • thread picker / replay from the map's run list (joinStream for live runs)
  • provenance from stream metadata (langgraph_node) instead of the narration prefix — needed for graphs that don't prefix
  • production-graph adoption: todos state key, and interrupt-node steering (current steering is stop-and-fold; graph-level interrupt() gives mid-node folding)
  • reasoning_content passthrough — verify the brain's llama.cpp reasoning-format flag surfaces thinking deltas
## Core implemented (deployed to :2027) All verified live against a phasewalk run in the browser: - **Rich transcript** — streaming markdown (react-markdown + gfm), highlight.js on the Sweetie-16 palette, diff fences as +/- cards; node **provenance chips** (parsed from the `[node]` narration prefix) - **Tool-call cards** — name + **MCP attribution chip** (`trog-mcp` / `websearch mcp` / `seat`), args summary, beat-locked spinner while pending, expandable args/result; calls paired to `ToolMessage`s by `tool_call_id` - **Todo list** — contract landed: `state.todos = [{text, status: pending|in_progress|done}]`; collapsible strip with count + current item. phasewalk maintains it across the walk - **Thinking foldout**, **run timeline** (per-node second chips from `updates` events — the fun-gate bounce is visible in the chips), **token meter** (latest turn usage when the model reports it), **quote-into-prompt**, autoscroll - **Steering** — input stays live mid-run; sending stops the stream and folds the line into the checkpointed thread as the next turn. Verified: steered at `alpha`, the follow-up walk kept checkpointed state (fun gate passed first try — the bounce disappeared) phasewalk exercises all of it GPU-free: todos, gates emitting real `check_phase` tool_calls + results, markdown narration. Structural test asserts both contracts. ## Remaining (next passes) - notifications (gate / error / done when backgrounded) + pixel chime - thread picker / replay from the map's run list (`joinStream` for live runs) - provenance from stream metadata (`langgraph_node`) instead of the narration prefix — needed for graphs that don't prefix - production-graph adoption: `todos` state key, and *interrupt-node* steering (current steering is stop-and-fold; graph-level `interrupt()` gives mid-node folding) - `reasoning_content` passthrough — verify the brain's llama.cpp reasoning-format flag surfaces thinking deltas
cmoriarty commented 2026-07-25 23:47:34 -04:00
Author
Owner
Copy link

Swappable brains shipped (feed's timeline strip replaced by the brain selector)

Design: llama-swap was exactly the right tool. brain/ builds trog-brain: the same pinned llama.cpp engine digest (the surprise-engine-swap lesson survives) fronted by llama-swap v242. One endpoint on :8090, the request's model field picks the brain, one model at a time, requests queued during loads. BRAIN_URL/BRAIN_MODEL contracts untouched; orchestrator brain bracketing untouched (stopping the container still frees GPUs); brain_idle grew a llama-swap-aware fallback (/running + /upstream/{m}/slots).

Roster (brain/llama-swap.yaml, all off the RAID): qwen3.6-27b (default — flags verbatim from the static brain, preloaded on startup) · qwen3.6-27b-q6 · qwen3.6-35b-a3b · qwen3-coder-next · qwen3-vl-8b · gemma-4-26b-a4b · gemma-4-31b.

Graphs: configurable.model overrides the default in hello + production (module llm stays default so the FakeLLM tests keep working). Studio submit rides it.

Studio: brain dropdown above the prompt — roster from /brain/v1/models, · loaded marker from /running, honest "swaps in on next send (~a minute to load)" hint when picking a cold one. Third same-origin proxy /brain.

Verified live end to end: default 27B answered through the proxy → model: qwen3-vl-8b swapped brains in 48s and answered → studio dropdown + hello graph swapped back to 27B (STUDIO BRAIN SWAP OK). Suite 118 passed.

Watch item: the llama-swap-aware brain_idle fallback hasn't seen a real seat cycle yet — first trog image … or drain after this deploy is the live test.

## Swappable brains shipped (feed's timeline strip replaced by the brain selector) **Design:** llama-swap was exactly the right tool. `brain/` builds `trog-brain`: the **same pinned llama.cpp engine digest** (the surprise-engine-swap lesson survives) fronted by llama-swap v242. One endpoint on :8090, the request's `model` field picks the brain, one model at a time, requests queued during loads. `BRAIN_URL`/`BRAIN_MODEL` contracts untouched; orchestrator brain bracketing untouched (stopping the container still frees GPUs); `brain_idle` grew a llama-swap-aware fallback (`/running` + `/upstream/{m}/slots`). **Roster** (brain/llama-swap.yaml, all off the RAID): `qwen3.6-27b` (default — flags verbatim from the static brain, preloaded on startup) · `qwen3.6-27b-q6` · `qwen3.6-35b-a3b` · `qwen3-coder-next` · `qwen3-vl-8b` · `gemma-4-26b-a4b` · `gemma-4-31b`. **Graphs:** `configurable.model` overrides the default in hello + production (module llm stays default so the FakeLLM tests keep working). Studio submit rides it. **Studio:** brain dropdown above the prompt — roster from `/brain/v1/models`, `· loaded` marker from `/running`, honest "swaps in on next send (~a minute to load)" hint when picking a cold one. Third same-origin proxy `/brain`. **Verified live end to end:** default 27B answered through the proxy → `model: qwen3-vl-8b` swapped brains in 48s and answered → studio dropdown + hello graph swapped back to 27B (`STUDIO BRAIN SWAP OK`). Suite 118 passed. **Watch item:** the llama-swap-aware `brain_idle` fallback hasn't seen a real seat cycle yet — first `trog image …` or drain after this deploy is the live test.
cmoriarty commented 2026-07-26 00:07:21 -04:00
Author
Owner
Copy link

Roster audit (4× Quadro RTX 5000, 64GB total) — deployed, every brain verified live

brain quant ctx verdict
qwen3.6-27b (default) UD-Q4_K_XL 262k unchanged — bake-off winner; Q6 would halve context for marginal gain
qwen3.6-35b-a3b UD-Q8_K_XL (38.5G, fetched) 131k upgraded from Q4 — 64GB affords near-lossless on the MoE; loads in ~15s, VRAM 11.5–14.3G/GPU with headroom
qwen3-coder-next UD-Q4_K_XL (49.6G) 64k Q4 IS the ceiling at 12.4G/GPU; verified loads + answers (~80s swap)
gemma-4-31b UD-Q4_K_XL 131k dense QAT beats 26B-A4B on quality; QAT is trained for q4 — Q4 ≈ lossless

Cut: qwen3-vl-8b, gemma-4-26b-a4b (not brain-grade), qwen3.6-27b-q6. The llama-swap config is the roster — the studio selector shows exactly these four; nothing else can be requested.

Bug found during verify: gemma-4's CLIP rejects --image-min-tokens 1024 (its image_max_pixels cap is below what 1024 min tokens implies) and llama-server exits at load. The flag is qwen-vision calibration; removed from the gemma entry, noted in the config.

Default restored and resident (qwen3.6-27b · ready).

## Roster audit (4× Quadro RTX 5000, 64GB total) — deployed, every brain verified live | brain | quant | ctx | verdict | |---|---|---|---| | `qwen3.6-27b` (default) | UD-Q4_K_XL | 262k | unchanged — bake-off winner; Q6 would halve context for marginal gain | | `qwen3.6-35b-a3b` | **UD-Q8_K_XL** (38.5G, fetched) | 131k | upgraded from Q4 — 64GB affords near-lossless on the MoE; loads in ~15s, VRAM 11.5–14.3G/GPU with headroom | | `qwen3-coder-next` | UD-Q4_K_XL (49.6G) | 64k | Q4 IS the ceiling at 12.4G/GPU; verified loads + answers (~80s swap) | | `gemma-4-31b` | UD-Q4_K_XL | 131k | dense QAT beats 26B-A4B on quality; QAT is trained for q4 — Q4 ≈ lossless | **Cut:** `qwen3-vl-8b`, `gemma-4-26b-a4b` (not brain-grade), `qwen3.6-27b-q6`. The llama-swap config is the roster — the studio selector shows exactly these four; nothing else can be requested. **Bug found during verify:** gemma-4's CLIP rejects `--image-min-tokens 1024` (its `image_max_pixels` cap is below what 1024 min tokens implies) and llama-server exits at load. The flag is qwen-vision calibration; removed from the gemma entry, noted in the config. Default restored and resident (`qwen3.6-27b · ready`).
cmoriarty commented 2026-07-26 00:23:20 -04:00
Author
Owner
Copy link

tok/s in the brain bar (deployed): output tokens per generation second for the last run, measured client-side from first streamed chunk to run end (excludes queueing and model-load, counts thinking tokens — honest across every roster brain). Verified live: 15.8 tok/s on the 27B default. Found and fixed en route: usage_metadata never reached the client — langchain-openai streams without include_usage by default, so the token meter had been silently dead; stream_usage=True on every brain client in hello + production feeds both readouts now (21↑ 661↓ alive).

**tok/s in the brain bar** (deployed): output tokens per generation second for the last run, measured client-side from first streamed chunk to run end (excludes queueing and model-load, counts thinking tokens — honest across every roster brain). Verified live: `15.8 tok/s` on the 27B default. Found and fixed en route: `usage_metadata` never reached the client — langchain-openai streams without `include_usage` by default, so the token meter had been silently dead; `stream_usage=True` on every brain client in hello + production feeds both readouts now (`21↑ 661↓` alive).
cmoriarty commented 2026-07-26 10:05:52 -04:00
Author
Owner
Copy link

Brain-swap status shipped (deployed): llama-swap /running state feeds both readouts — topbar activity shows swapping brain · <model> first while any model is non-ready, feed working line says swapping brain during the load, · loaded marker only counts ready models, roster poll tightens to 3s mid-run. Captured live through a 27B→gemma swap: swapping brain · qwen3.6-27b (unload) → swapping brain · gemma-4-31b · run · call_brain (load) → run · call_brain → idle.

**Brain-swap status shipped** (deployed): llama-swap /running state feeds both readouts — topbar activity shows `swapping brain · <model>` first while any model is non-ready, feed working line says `swapping brain` during the load, `· loaded` marker only counts ready models, roster poll tightens to 3s mid-run. Captured live through a 27B→gemma swap: `swapping brain · qwen3.6-27b` (unload) → `swapping brain · gemma-4-31b · run · call_brain` (load) → `run · call_brain` → `idle`.
cmoriarty commented 2026-07-26 10:46:14 -04:00
Author
Owner
Copy link

Final #8 round (deployed) — games, sessions, provenance, live reasoning

  • Games — top-bar selector; scratch is the ungrouped default; + new game… creates a stub inline (real wizard = #25). Thread metadata.game is the source of truth; stubs live in localStorage until their first session exists.
  • Brain sessions — per game+graph, the feed's session picker lists the game's threads; switching loads that thread's full history (replay for free); new sessions are stamped with the game at birth and re-stamped after every run — aegra rewrites thread metadata on run start and can clobber a stamp that landed first (caught live, fixed).
  • Provenance from stream metadata — chips now read langgraph_node from the stream itself (getMessagesMetadata().streamMetadata), so unprefixed graphs get chips too; verified call_brain on hello.
  • Live reasoning — ChatDeepSeek replaces ChatOpenAI as the brain client (same wire; captures streamed reasoning_content, which base ChatOpenAI silently drops — root cause of the foldout never firing). The brain's thinking now streams into the feed foldout in real time while it works, and never pollutes content/docs. Verified mid-run at 15s into a think.
  • Notifications: dropped per review. Production-graph todos + interrupt steering: moved to #11.

That empties the ticket as scoped + reviewed.

## Final #8 round (deployed) — games, sessions, provenance, live reasoning - **Games** — top-bar selector; `scratch` is the ungrouped default; `+ new game…` creates a stub inline (real wizard = #25). Thread `metadata.game` is the source of truth; stubs live in localStorage until their first session exists. - **Brain sessions** — per game+graph, the feed's session picker lists the game's threads; switching **loads that thread's full history** (replay for free); new sessions are stamped with the game at birth and re-stamped after every run — aegra rewrites thread metadata on run start and can clobber a stamp that landed first (caught live, fixed). - **Provenance from stream metadata** — chips now read `langgraph_node` from the stream itself (`getMessagesMetadata().streamMetadata`), so unprefixed graphs get chips too; verified `call_brain` on hello. - **Live reasoning** — ChatDeepSeek replaces ChatOpenAI as the brain client (same wire; captures streamed `reasoning_content`, which base ChatOpenAI silently drops — root cause of the foldout never firing). The brain's thinking now streams into the feed foldout in real time while it works, and never pollutes content/docs. Verified mid-run at 15s into a think. - Notifications: dropped per review. Production-graph todos + interrupt steering: moved to #11. That empties the ticket as scoped + reviewed.
cmoriarty commented 2026-07-26 11:32:41 -04:00
Author
Owner
Copy link

Closing summary

The feed went from token stream to a coding-agent-grade window into the studio, plus infrastructure it pulled in along the way. All deployed to :2027; running containers verified equal to latest images; suite 118 passed.

Chat/feed: streaming markdown + syntax highlighting · tool-call cards with MCP attribution and live spinners · the graph's todo list (state.todos contract) · node-provenance chips from stream metadata · live-streaming reasoning foldout (ChatDeepSeek brain client — base ChatOpenAI drops reasoning_content) · steering mid-run (stop-and-fold, checkpointed state proven to survive) · token meter + tok/s.

Games + brain sessions: top-bar game selector (thread metadata.game = truth, scratch for the unstamped, inline stub creation until #25's wizard) · per-game session picker with full history load on switch · birth + post-run stamping (aegra metadata-clobber race caught live).

Swappable brains (grew out of this ticket): brain/ = llama-swap v242 fronting the pinned llama.cpp engine · curated 4-model roster audited for the 4× RTX 5000 (27B default @262k, 35B-A3B UD-Q8_K_XL, Coder-Next at the VRAM ceiling, gemma-4-31B QAT) · every brain verified on the metal · graphs honor configurable.model · brain bar: selector, loaded marker, swap progress, tok/s · topbar shows swapping brain · <model> through loads.

Cut/moved: notifications (review call) · quote buttons (review call) · timeline strip (superseded by brain bar) · production-graph todos + interrupt steering → #11.

Contracts this ticket established: state.todos, configurable.model, metadata.game, the [node]-or-metadata provenance rule, stream_usage=True on every brain client. phasewalk exercises all of it GPU-free.

Watch item that remains: seats brain_idle's llama-swap fallback still wants a real seat cycle.

## Closing summary The feed went from token stream to a **coding-agent-grade window into the studio**, plus infrastructure it pulled in along the way. All deployed to :2027; running containers verified equal to latest images; suite 118 passed. **Chat/feed:** streaming markdown + syntax highlighting · tool-call cards with MCP attribution and live spinners · the graph's todo list (`state.todos` contract) · node-provenance chips from stream metadata · **live-streaming reasoning foldout** (ChatDeepSeek brain client — base ChatOpenAI drops `reasoning_content`) · steering mid-run (stop-and-fold, checkpointed state proven to survive) · token meter + tok/s. **Games + brain sessions:** top-bar game selector (thread `metadata.game` = truth, `scratch` for the unstamped, inline stub creation until #25's wizard) · per-game session picker with full history load on switch · birth + post-run stamping (aegra metadata-clobber race caught live). **Swappable brains** (grew out of this ticket): `brain/` = llama-swap v242 fronting the pinned llama.cpp engine · curated 4-model roster audited for the 4× RTX 5000 (27B default @262k, 35B-A3B **UD-Q8_K_XL**, Coder-Next at the VRAM ceiling, gemma-4-31B QAT) · every brain verified on the metal · graphs honor `configurable.model` · brain bar: selector, loaded marker, swap progress, tok/s · topbar shows `swapping brain · <model>` through loads. **Cut/moved:** notifications (review call) · quote buttons (review call) · timeline strip (superseded by brain bar) · production-graph todos + interrupt steering → #11. **Contracts this ticket established:** `state.todos`, `configurable.model`, `metadata.game`, the `[node]`-or-metadata provenance rule, `stream_usage=True` on every brain client. phasewalk exercises all of it GPU-free. Watch item that remains: seats brain_idle's llama-swap fallback still wants a real seat cycle.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
feat/11-walking-skeleton
No results found.
Labels
Clear labels   
bug
Something is not working

  
duplicate
This issue or pull request already exists

  
enhancement
New feature

  
help wanted
Need some help

  
invalid
Something is wrong

  
question
More information is needed

  
wontfix
This won't be fixed

No labels
bug
duplicate
enhancement
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
cmoriarty/trog#8
No description provided.