Top bar on the Free profile: name the model in use, count slots the router can serve, and say what the quota cell means #194

Closed
opened 2026-10-18 01:45:42 -04:00 by cmoriarty · 3 comments
Owner

Seen on production during run 76, the first run on the Free profile (97242c8). The top bar read:

INFERENCE STATUS SLOTS FREE QUOTA
Free Running 7/6 reader 100% · resets 03:00

The operator's words:

Its great that it shows Free, but it should show the current provider-model being used. Also, it seems like max slots is set to 6, but we seem to have the ability to have more parallel agents. Maybe when there are more than one running it just says multiple. Also, I don't really understand what the FREE QUOTA cell is showing.

What each cell does today

  • Inference shows the backend's label. On the Free profile that is the profile's name, "Free", although every turn is served by one provider's model, and Braid already records which one served each turn.
  • Slots divides the requests in flight by max_concurrent_upstream (6), the constant measured for vLLM's --max-num-seqs 6 (backend_capacity in src/osf/api/artifacts.py reads a server's own count only for llama.cpp). The router fans requests out over many providers, so 6 is not its capacity, and "7/6" reads as over-full when nothing is.
  • Free quota names the tier with the least share left across its models (least in GET /api/free/quota), that share, and that tier's next reset in local time. "reader 100% · resets 03:00" means even the lowest tier is untouched, and its next reset is Gemini's day (midnight Pacific). None of that is said.

To do

  1. Inference: name the provider and model serving the running step (e.g. "OVH · Qwen3.5-397B"), and "multiple" when the running steps are on different models, with the list in its hover.
  2. Slots: on the Free profile, a capacity the router really has (or none, just the count in flight), so the cell never reads over-full because of vLLM's constant.
  3. Free quota: words that say what the cell is, without needing the explanation above.
Seen on production during run 76, the first run on the Free profile (97242c8). The top bar read: | INFERENCE | STATUS | SLOTS | FREE QUOTA | | --- | --- | --- | --- | | Free | Running | 7/6 | reader 100% · resets 03:00 | The operator's words: > Its great that it shows Free, but it should show the current provider-model being used. Also, it seems like max slots is set to 6, but we seem to have the ability to have more parallel agents. Maybe when there are more than one running it just says multiple. Also, I don't really understand what the FREE QUOTA cell is showing. **What each cell does today** - **Inference** shows the backend's label. On the Free profile that is the profile's name, "Free", although every turn is served by one provider's model, and Braid already records which one served each turn. - **Slots** divides the requests in flight by `max_concurrent_upstream` (6), the constant measured for vLLM's `--max-num-seqs 6` (`backend_capacity` in `src/osf/api/artifacts.py` reads a server's own count only for llama.cpp). The router fans requests out over many providers, so 6 is not its capacity, and "7/6" reads as over-full when nothing is. - **Free quota** names the tier with the least share left across its models (`least` in `GET /api/free/quota`), that share, and that tier's next reset in local time. "reader 100% · resets 03:00" means even the lowest tier is untouched, and its next reset is Gemini's day (midnight Pacific). None of that is said. **To do** 1. Inference: name the provider and model serving the running step (e.g. "OVH · Qwen3.5-397B"), and "multiple" when the running steps are on different models, with the list in its hover. 2. Slots: on the Free profile, a capacity the router really has (or none, just the count in flight), so the cell never reads over-full because of vLLM's constant. 3. Free quota: words that say what the cell is, without needing the explanation above.
Author
Owner

Shipped in ead9f7e (change free-top-bar, merged as 1330513). Seen on production during run 77: INFERENCE kilo/beam-free (hover: the step and run), AGENTS 1 (hover: the Free router has no slot limit), FREE QUOTA lowest: strong tier 64% left, with the four-line hover (what a tier is, how it is counted, the next refill and whose limit it is, every tier's share). vLLM, llama.cpp and Anthropic top bars unchanged. The extras are #199. free-top-bar must archive after free-profile (#164).

Left open: the OpenSpec change is not archived yet, and its tasks are not ticked against production. Deferred when the session wrapped up (2026-10-18).

Shipped in ead9f7e (change `free-top-bar`, merged as 1330513). Seen on production during run 77: INFERENCE `kilo/beam-free` (hover: the step and run), AGENTS `1` (hover: the Free router has no slot limit), FREE QUOTA `lowest: strong tier 64% left`, with the four-line hover (what a tier is, how it is counted, the next refill and whose limit it is, every tier's share). vLLM, llama.cpp and Anthropic top bars unchanged. The extras are #199. `free-top-bar` must archive after `free-profile` (#164). Left open: the OpenSpec change is not archived yet, and its tasks are not ticked against production. Deferred when the session wrapped up (2026-10-18).
Author
Owner

Closing: verified on production (run 77, ead9f7e), see the comment above. The OpenSpec change free-top-bar is not archived yet: it modifies requirements that free-profile (#164) adds, so it archives right after that change. The extras are #199.

Closing: verified on production (run 77, ead9f7e), see the comment above. The OpenSpec change `free-top-bar` is not archived yet: it modifies requirements that `free-profile` (#164) adds, so it archives right after that change. The extras are #199.
Author
Owner

Archived in OpenSpec as free-top-bar (5a953a0).

Archived in OpenSpec as `free-top-bar` (5a953a0).
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#194
No description provided.