Setting up a backend means guessing its numbers: Connect first, then fill the model's context, output reservation and effort from what the server reports #162

Closed
opened 2026-10-09 00:31:37 -04:00 by cmoriarty · 1 comment
Owner

The Settings page asks for a backend's URL, model, context window, output reservation and reasoning effort all at once, before Braid has asked the server anything. For a new provider an operator has no idea what to enter: Anthropic's URL is always the same, and the window and output limits are published by the server itself.

What the servers report, read on 2026-10-08: Anthropic's GET /v1/models gives each model's display_name, max_input_tokens (1,000,000 for the Claude 5.x models), max_tokens (128,000), the effort levels it supports, and deprecated_at. vLLM's model list carries max_model_len (to confirm on trogdor). No server recommends a reasoning effort.

What it needs

  • A default base URL per kind where one exists: Anthropic → https://api.anthropic.com/v1. Self-hosted kinds keep a placeholder.
  • Test connection becomes Connect. Until Connect has listed the server's models, only kind, URL and key are enabled; the model, window, output reservation and effort fields stay disabled.
  • The model becomes a picker of the listed models, with their display names. Choosing one fills its defaults:
    • context window: what the server reports (max_input_tokens, max_model_len);
    • output reservation and reasoning effort: from a small built-in table of models Braid knows (Claude 5.x, qwen3.8), from Anthropic's guidance and Braid's own measurements (#60, #124, #136). Any other model gets min(its max output, 32,000), opencode's cap, and medium.
    • Each value is shown as the model's default and stays editable; the effort offers only the levels the model supports.
  • Builds on the backend profiles (#161): Connect sets up a profile.

Done when

Adding the Claude API takes the kind, the key, Connect and a model pick, with every number filled from the model; and trogdor's profile fills its window from vLLM's model list.

The Settings page asks for a backend's URL, model, context window, output reservation and reasoning effort all at once, before Braid has asked the server anything. For a new provider an operator has no idea what to enter: Anthropic's URL is always the same, and the window and output limits are published by the server itself. What the servers report, read on 2026-10-08: Anthropic's `GET /v1/models` gives each model's `display_name`, `max_input_tokens` (1,000,000 for the Claude 5.x models), `max_tokens` (128,000), the effort levels it supports, and `deprecated_at`. vLLM's model list carries `max_model_len` (to confirm on trogdor). No server recommends a reasoning effort. ## What it needs - **A default base URL per kind** where one exists: Anthropic → `https://api.anthropic.com/v1`. Self-hosted kinds keep a placeholder. - **Test connection becomes Connect.** Until Connect has listed the server's models, only kind, URL and key are enabled; the model, window, output reservation and effort fields stay disabled. - **The model becomes a picker** of the listed models, with their display names. Choosing one fills its defaults: - context window: what the server reports (`max_input_tokens`, `max_model_len`); - output reservation and reasoning effort: from a small built-in table of models Braid knows (Claude 5.x, qwen3.8), from Anthropic's guidance and Braid's own measurements (#60, #124, #136). Any other model gets `min(its max output, 32,000)`, opencode's cap, and `medium`. - Each value is shown as the model's default and stays editable; the effort offers only the levels the model supports. - Builds on the backend profiles (#161): Connect sets up a profile. ## Done when Adding the Claude API takes the kind, the key, Connect and a model pick, with every number filled from the model; and trogdor's profile fills its window from vLLM's model list.
Author
Owner

Shipped in 649dc00 (deployed; new boot id). Change archived as openspec/changes/archive/2026-10-09-backend-connect.

What shipped

  • Connect replaces Test connection and reports each model's details: display name, context window and maximum output when the server reports them, the efforts it supports, and Braid's recommended reservation and effort. src/osf/models.py reads Anthropic's shape (max_input_tokens, max_tokens, capabilities.effort), vLLM's max_model_len, and id-only lists. The answer keeps models and adds model_details.
  • A new backend, or one whose kind or URL changed, keeps its model, window, reservation and effort locked until Connect lists the models; Set up without connecting covers a server that cannot. Saved backends open unlocked.
  • Choosing Anthropic fills https://api.anthropic.com/v1 unless a URL was typed.
  • The model is picked from the list (display name and id, plus "Another model…"). A pick fills the window from the server and the reservation and effort from a small table (Claude 5.5: 32,000 and medium; qwen3.8-27b: 20,480 and medium), else min(max output, 32,000) and medium. A field edited away shows the model's default; the effort offers only supported levels.

Checked

  • Live against the Claude API (model list only, no spend), key read from the operator's key file by path: 14 models listed; Sonnet 5.5 and Opus 5.5 filled 1,000,000 / 32,000 / medium; Sonnet 4.5, not in the table, filled the server's 200,000 and the rule's 32,000.
  • trogdor's vLLM answers "max_model_len": 524288; Connect from production after the deploy read it as a 524,288 window with 20,480 and medium.
  • Full lane green (backend, 190 browser specs, 13 simulated runs).
Shipped in 649dc00 (deployed; new boot id). Change archived as `openspec/changes/archive/2026-10-09-backend-connect`. **What shipped** - **Connect** replaces Test connection and reports each model's details: display name, context window and maximum output when the server reports them, the efforts it supports, and Braid's recommended reservation and effort. `src/osf/models.py` reads Anthropic's shape (`max_input_tokens`, `max_tokens`, `capabilities.effort`), vLLM's `max_model_len`, and id-only lists. The answer keeps `models` and adds `model_details`. - A new backend, or one whose kind or URL changed, keeps its model, window, reservation and effort locked until Connect lists the models; **Set up without connecting** covers a server that cannot. Saved backends open unlocked. - Choosing Anthropic fills `https://api.anthropic.com/v1` unless a URL was typed. - The model is picked from the list (display name and id, plus "Another model…"). A pick fills the window from the server and the reservation and effort from a small table (Claude 5.5: 32,000 and `medium`; `qwen3.8-27b`: 20,480 and `medium`), else `min(max output, 32,000)` and `medium`. A field edited away shows the model's default; the effort offers only supported levels. **Checked** - Live against the Claude API (model list only, no spend), key read from the operator's key file by path: 14 models listed; Sonnet 5.5 and Opus 5.5 filled 1,000,000 / 32,000 / `medium`; Sonnet 4.5, not in the table, filled the server's 200,000 and the rule's 32,000. - trogdor's vLLM answers `"max_model_len": 524288`; Connect from production after the deploy read it as a 524,288 window with 20,480 and `medium`. - Full lane green (backend, 190 browser specs, 13 simulated runs).
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#162
No description provided.