Runs on the Claude API pay full price for every turn's context: add a native Anthropic backend so opencode's prompt caching applies #155

Closed
opened 2026-10-08 22:36:32 -04:00 by cmoriarty · 1 comment
Owner

Braid can already point a run at the Claude API through Anthropic's OpenAI-compatible endpoint (kind OpenAI-compatible, URL https://api.anthropic.com/v1, a key file). That layer does not support prompt caching (Anthropic's OpenAI SDK compatibility docs, "Important OpenAI compatibility limitations"), so every agent turn pays full input price for the whole conversation so far. Braid's steps run long, growing contexts, so most of the spend would be resending the same prefix.

The native Messages API (/v1/messages) caches, and opencode already does the caching itself: with @ai-sdk/anthropic as the provider (or a model id containing claude), it marks the first two system messages and the last two messages cacheControl: ephemeral on every request. Read from the opencode 1.18.34 binary; the pinned 1.18.21 still needs checking. The Anthropic client is bundled in the binary, so nothing new is installed.

What it needs

  1. A backend kind Anthropic in BACKEND_KINDS (src/osf/settings.py) and the settings page.
  2. For that kind, render_run_config (src/osf/agent/server.py) renders the provider with npm: "@ai-sdk/anthropic", for the main backend and the planning one; the key stays a {file:...} reference.
  3. Headers per kind: auth_headers sends Authorization: Bearer, but Anthropic's GET /v1/models wants x-api-key and anthropic-version. It is used by the engine probe (engine/runtime.py), the settings page's Test (api/settings.py) and api/artifacts.py. Without this the probe fails and no agent step is admitted.
  4. Reasoning effort maps to Anthropic's own effort option for this kind; no chat_template_kwargs, no thinking budget, and no nothink/brief variants (Claude Opus 5.5 rejects disabling thinking with a 400).

Done when

  • Unit tests cover the rendered config and the probe headers for the new kind.
  • One small live run on the Claude API, under a workspace spend limit, finishes, and its steps' token counts show cache reads on most requests.
  • Checked: the pinned opencode caches the same way; the console's token view handles cache-read counts; how often Braid's handoff and re-prompt turns break the cached prefix.

The fake model in the simulated runs speaks only the OpenAI API, so the live run is the end-to-end check unless a fake /v1/messages is worth adding.

Braid can already point a run at the Claude API through Anthropic's OpenAI-compatible endpoint (kind `OpenAI-compatible`, URL `https://api.anthropic.com/v1`, a key file). That layer does not support prompt caching (Anthropic's OpenAI SDK compatibility docs, "Important OpenAI compatibility limitations"), so every agent turn pays full input price for the whole conversation so far. Braid's steps run long, growing contexts, so most of the spend would be resending the same prefix. The native Messages API (`/v1/messages`) caches, and opencode already does the caching itself: with `@ai-sdk/anthropic` as the provider (or a model id containing `claude`), it marks the first two system messages and the last two messages `cacheControl: ephemeral` on every request. Read from the opencode 1.18.34 binary; the pinned 1.18.21 still needs checking. The Anthropic client is bundled in the binary, so nothing new is installed. ## What it needs 1. A backend kind `Anthropic` in `BACKEND_KINDS` (`src/osf/settings.py`) and the settings page. 2. For that kind, `render_run_config` (`src/osf/agent/server.py`) renders the provider with `npm: "@ai-sdk/anthropic"`, for the main backend and the planning one; the key stays a `{file:...}` reference. 3. Headers per kind: `auth_headers` sends `Authorization: Bearer`, but Anthropic's `GET /v1/models` wants `x-api-key` and `anthropic-version`. It is used by the engine probe (`engine/runtime.py`), the settings page's Test (`api/settings.py`) and `api/artifacts.py`. Without this the probe fails and no agent step is admitted. 4. Reasoning effort maps to Anthropic's own effort option for this kind; no `chat_template_kwargs`, no thinking budget, and no `nothink`/`brief` variants (Claude Opus 5.5 rejects disabling thinking with a 400). ## Done when - Unit tests cover the rendered config and the probe headers for the new kind. - One small live run on the Claude API, under a workspace spend limit, finishes, and its steps' token counts show cache reads on most requests. - Checked: the pinned opencode caches the same way; the console's token view handles cache-read counts; how often Braid's handoff and re-prompt turns break the cached prefix. The fake model in the simulated runs speaks only the OpenAI API, so the live run is the end-to-end check unless a fake `/v1/messages` is worth adding.
Author
Owner

Shipped in ecf013e (deployed; production serves it, new boot id). Change archived as openspec/changes/archive/2026-10-08-anthropic-backend.

What shipped

  • A backend kind Anthropic, for the main and the planning backend. A run on it gets opencode's own Anthropic client, which caches the prompt: captured from opencode 1.18.35, every request marks the system block and the last message cache_control: ephemeral. The key is sent as x-api-key with anthropic-version, by the run, Test connection and the engine probe.
  • The reasoning effort is sent as Anthropic's own (high included); the handoff and the cut-off re-prompt think at low there, since current Claude models refuse a request with thinking off.
  • The context gauge counts cached input on every backend. opencode's tokens.input is only the uncached part; without this the gauge read 4 for a 16,680-token turn and would never have handed a step off on Claude.
  • opencode's five-minute request timeouts (new in 1.18.27) are off for self-hosted servers, as every run had them before; hosted kinds keep them.
  • Pins: opencode 1.18.35, OpenSpec CLI 1.14.1 (skills regenerated with openspec update), the uv image 0.12.24, lockfiles refreshed within their ranges. Playwright and its MCP stay at 1.63.0 / 0.0.80: no MCP release is built on Playwright 1.64.0.

Live check, Claude Sonnet 5.5, one quick-fix run from a local stack: both agent steps succeeded, lanes green on the first run, no model error or 429. From the second turn of a step on, about 97% of each request was a cache read; 70% of all input over the two short steps. $0.10 against $0.21 uncached.

Deferred: #156 (long requirements under OpenSpec 1.14.1's strict check), #157 (Playwright 1.64 with its MCP), #158 (TypeScript 7, virtua 0.53, @types/node 26), #159 (Node and Python base images).

Not yet checked: a trogdor run on the new build. The model server is unreachable from production and from the laptop right now (unrelated to this deploy); the check is to start one run and see its first agent step begin.

To use it: Settings → Kind Anthropic, URL https://api.anthropic.com/v1, a model such as claude-sonnet-5-5, and a key file holding a workspace-scoped key.

Shipped in ecf013e (deployed; production serves it, new boot id). Change archived as `openspec/changes/archive/2026-10-08-anthropic-backend`. **What shipped** - A backend kind **Anthropic**, for the main and the planning backend. A run on it gets opencode's own Anthropic client, which caches the prompt: captured from opencode 1.18.35, every request marks the system block and the last message `cache_control: ephemeral`. The key is sent as `x-api-key` with `anthropic-version`, by the run, Test connection and the engine probe. - The reasoning effort is sent as Anthropic's own (`high` included); the handoff and the cut-off re-prompt think at `low` there, since current Claude models refuse a request with thinking off. - **The context gauge counts cached input on every backend.** opencode's `tokens.input` is only the uncached part; without this the gauge read 4 for a 16,680-token turn and would never have handed a step off on Claude. - opencode's five-minute request timeouts (new in 1.18.27) are off for self-hosted servers, as every run had them before; hosted kinds keep them. - Pins: opencode 1.18.35, OpenSpec CLI 1.14.1 (skills regenerated with `openspec update`), the uv image 0.12.24, lockfiles refreshed within their ranges. Playwright and its MCP stay at 1.63.0 / 0.0.80: no MCP release is built on Playwright 1.64.0. **Live check**, Claude Sonnet 5.5, one quick-fix run from a local stack: both agent steps succeeded, lanes green on the first run, no model error or 429. From the second turn of a step on, about 97% of each request was a cache read; 70% of all input over the two short steps. $0.10 against $0.21 uncached. **Deferred:** #156 (long requirements under OpenSpec 1.14.1's strict check), #157 (Playwright 1.64 with its MCP), #158 (TypeScript 7, virtua 0.53, @types/node 26), #159 (Node and Python base images). **Not yet checked:** a trogdor run on the new build. The model server is unreachable from production and from the laptop right now (unrelated to this deploy); the check is to start one run and see its first agent step begin. To use it: Settings → Kind **Anthropic**, URL `https://api.anthropic.com/v1`, a model such as `claude-sonnet-5-5`, and a key file holding a workspace-scoped key.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#155
No description provided.