A FREE profile: run Braid on free model APIs, routing between them as each hits its limit and handing the work off to the next #164

Closed
opened 2026-10-09 03:01:07 -04:00 by cmoriarty · 1 comment
Owner

Braid needs either a local GPU (trogdor) or a paid API key (the Claude API, #155). Someone with neither cannot try it. Several providers offer free tiers of capable models, but each is narrow: a few requests a minute, a daily token or request cap, a smaller context window. No one of them can carry a whole run. Together they might.

This issue is to investigate a FREE profile: one backend profile (#161) that is really a router over a list of free providers. It uses one until it is throttled or exhausted, then hands the work off to the next and carries on, so a run can finish on free capacity alone.

Questions the investigation answers

  1. Which free tiers exist, and what they allow, measured or read from each provider's current documentation and terms, not assumed:
    • the models offered, their context windows and maximum output;
    • their limits: requests per minute and per day, tokens per minute and per day, and how a provider signals each one (status code, headers, error body, reset time);
    • their API shape: OpenAI-compatible or their own, and tool calling, which Braid's agents depend on;
    • their terms: whether automated or agentic use is allowed, and whether prompts are used for training, which matters because a run sends a repository's code.
      Candidates to check, not a list to adopt: Google's Gemini API free tier, Groq, Cerebras, Mistral, OpenRouter's free models, GitHub Models, Cloudflare Workers AI, Hugging Face inference.
  2. Can they do the work? Braid's steps need reliable tool calls over long sessions. Run the simulated pipelines and one small real quick fix on each candidate, and record which can complete a step at all.
  3. Where can Braid switch?
    • Between steps is the easy case: each step already names its provider and model (_model_spec), and a run's opencode config can declare several providers.
    • Mid-step is the hard case. When a provider stops answering partway through a session, the step must continue on another model. Braid already hands a session off when its context runs out (#60): the agent writes a handoff note and a fresh session continues from it. Can the same mechanism carry work from one provider to another? What does the handoff turn itself run on, given the exhausted provider cannot write it?
    • What is lost when switching: prompt caching (per provider), thinking blocks (bound to their model), and a window that shrinks when the next model's is smaller than the session.
  4. The router's policy:
    • the order to try providers in (capability first, or remaining quota first);
    • how it learns that a limit was hit (a 429, headers, a daily cap that resets at a known time);
    • whether it waits for a per-minute limit to reset or moves on;
    • what a run does when every provider is exhausted: pause until one resets, and say so in the run view, rather than fail.
  5. What the operator sees:
    • which provider each step and turn ran on;
    • how much free quota each provider has left;
    • one key field per provider within the FREE profile, under #160's rules (write-only, bound to its host).

Done when

The investigation reports which free providers can run Braid's steps, with their measured limits and terms. It recommends a design (between-step only, or mid-step handoff too) with its known costs. It is backed by one real run that finished on free providers alone, or by evidence of why that is not yet possible. The design then gets its own OpenSpec change.

Braid needs either a local GPU (trogdor) or a paid API key (the Claude API, #155). Someone with neither cannot try it. Several providers offer free tiers of capable models, but each is narrow: a few requests a minute, a daily token or request cap, a smaller context window. No one of them can carry a whole run. Together they might. This issue is to **investigate** a FREE profile: one backend profile (#161) that is really a router over a list of free providers. It uses one until it is throttled or exhausted, then hands the work off to the next and carries on, so a run can finish on free capacity alone. ## Questions the investigation answers 1. **Which free tiers exist, and what they allow**, measured or read from each provider's current documentation and terms, not assumed: - the models offered, their context windows and maximum output; - their limits: requests per minute and per day, tokens per minute and per day, and how a provider signals each one (status code, headers, error body, reset time); - their API shape: OpenAI-compatible or their own, and tool calling, which Braid's agents depend on; - their terms: whether automated or agentic use is allowed, and whether prompts are used for training, which matters because a run sends a repository's code. Candidates to check, not a list to adopt: Google's Gemini API free tier, Groq, Cerebras, Mistral, OpenRouter's free models, GitHub Models, Cloudflare Workers AI, Hugging Face inference. 2. **Can they do the work?** Braid's steps need reliable tool calls over long sessions. Run the simulated pipelines and one small real quick fix on each candidate, and record which can complete a step at all. 3. **Where can Braid switch?** - *Between steps* is the easy case: each step already names its provider and model (`_model_spec`), and a run's opencode config can declare several providers. - *Mid-step* is the hard case. When a provider stops answering partway through a session, the step must continue on another model. Braid already hands a session off when its context runs out (#60): the agent writes a handoff note and a fresh session continues from it. Can the same mechanism carry work from one provider to another? What does the handoff turn itself run on, given the exhausted provider cannot write it? - What is lost when switching: prompt caching (per provider), thinking blocks (bound to their model), and a window that shrinks when the next model's is smaller than the session. 4. **The router's policy**: - the order to try providers in (capability first, or remaining quota first); - how it learns that a limit was hit (a 429, headers, a daily cap that resets at a known time); - whether it waits for a per-minute limit to reset or moves on; - what a run does when every provider is exhausted: pause until one resets, and say so in the run view, rather than fail. 5. **What the operator sees**: - which provider each step and turn ran on; - how much free quota each provider has left; - one key field per provider within the FREE profile, under #160's rules (write-only, bound to its host). ## Done when The investigation reports which free providers can run Braid's steps, with their measured limits and terms. It recommends a design (between-step only, or mid-step handoff too) with its known costs. It is backed by one real run that finished on free providers alone, or by evidence of why that is not yet possible. The design then gets its own OpenSpec change.
Author
Owner

Archived in OpenSpec as free-profile (5a953a0). Shipped across 2026-10-17/18 (production c1228b4): the Free profile with a GoModel router, ten providers (NVIDIA opt-in, Alibaba opt-in, Cloudflare, Cohere direct, Gemini, OpenRouter, Z.ai, Inception, OVH, Kilo, Zen), tiers and chains from probes, quota ledger and parking, and the top bar (#194). Task 13.2 is recorded from production runs 76, 77 and 78 on free providers alone: 76 failed on a dry chain (fixed, #196), 77 was stopped for #201, 78 completed apply groups 1-4 and 6 with steps pausing on spent quota, refused sessions moving (#200) and stalled models put behind (#195), and could not finish mainly because of #202. What is left is #205. Closing.

Archived in OpenSpec as `free-profile` (5a953a0). Shipped across 2026-10-17/18 (production c1228b4): the Free profile with a GoModel router, ten providers (NVIDIA opt-in, Alibaba opt-in, Cloudflare, Cohere direct, Gemini, OpenRouter, Z.ai, Inception, OVH, Kilo, Zen), tiers and chains from probes, quota ledger and parking, and the top bar (#194). Task 13.2 is recorded from production runs 76, 77 and 78 on free providers alone: 76 failed on a dry chain (fixed, #196), 77 was stopped for #201, 78 completed apply groups 1-4 and 6 with steps pausing on spent quota, refused sessions moving (#200) and stalled models put behind (#195), and could not finish mainly because of #202. What is left is #205. Closing.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#164
No description provided.