Inference page on backends that report no load; Anthropic's slot count; the Free quota figure without minute and hour limits #199

Closed
opened 2026-10-18 03:08:45 -04:00 by cmoriarty · 3 comments
Owner

Follow-up to #194 (the top bar on the Free profile). The proposal for #194 (free-top-bar) kept to the operator's three cells, and these came up while writing it:

  1. The Inference page draws rows a backend cannot feed. On llama.cpp, SGLang, Anthropic and Free it draws the All requests, KV cache and Reused work rows, and a red "not answering" for a server that is simply not vLLM. Today that is a bug on every non-vLLM backend, not only Free. Draw only what the backend reports.
  2. Name each running step's model in its row on the Inference page, as #194 does in the top bar's hover, and read a step held for spent free quota as "waiting for free quota".
  3. Anthropic's 6 slots. As on Free, nothing enforces max_concurrent_upstream on the Anthropic profile, whose limits are rates. Give it the same "no slot total" rule (capacity_source: none).
  4. The Free quota figure counts minute and hour limits, so it jumps during a run and refills before anyone can act. Count only daily, monthly and one-time limits, word the cell reader tier 99% left today, and put the refill time back in the cell. Minute and hour limits still hold steps, which the cell shows as Paused until ….

Each is small. 1 and 3 are bugs; 2 and 4 are choices for the operator.

Follow-up to #194 (the top bar on the Free profile). The proposal for #194 (`free-top-bar`) kept to the operator's three cells, and these came up while writing it: 1. **The Inference page draws rows a backend cannot feed.** On llama.cpp, SGLang, Anthropic and Free it draws the All requests, KV cache and Reused work rows, and a red "not answering" for a server that is simply not vLLM. Today that is a bug on every non-vLLM backend, not only Free. Draw only what the backend reports. 2. **Name each running step's model in its row on the Inference page,** as #194 does in the top bar's hover, and read a step held for spent free quota as "waiting for free quota". 3. **Anthropic's 6 slots.** As on Free, nothing enforces `max_concurrent_upstream` on the Anthropic profile, whose limits are rates. Give it the same "no slot total" rule (`capacity_source: none`). 4. **The Free quota figure counts minute and hour limits,** so it jumps during a run and refills before anyone can act. Count only daily, monthly and one-time limits, word the cell `reader tier 99% left today`, and put the refill time back in the cell. Minute and hour limits still hold steps, which the cell shows as `Paused until …`. Each is small. 1 and 3 are bugs; 2 and 4 are choices for the operator.
Author
Owner

Point 1 (rows a backend can't feed) is now part of #289, the Inference page redesign per backend.

Point 1 (rows a backend can't feed) is now part of #289, the Inference page redesign per backend.
Author
Owner

Checked on production after #301 (5355515), with run 144 on Free. Two of the three points work; point 2 doesn't, so this stays open.

  • Point 3, Anthropic's slots: works. With the Anthropic profile active and no Parallel sessions, /api/slots and its history report capacity: null, capacity_source: none, where they read six before.
  • Point 4, the quota figure: works. The top bar reads FREE QUOTA all tiers 100% left today. During the run, beam-free had used 7 of its 200 requests that hour, and every tier still read 100% left: hour and minute limits no longer count.
  • Point 2, each step's model on the Inference page: not visible on Free. The label went into the slot rows, and #289 hides those on hosted backends, so the Free Inference page shows the quota tiers and the Agents chart but no row per running step. Fix: draw the running steps' rows on hosted backends without the slot tiles. Note for #309: a Free step's serving carries a name too (kilo/beam-free), so "on Free" must come from the profile, not from serving.name.
**Checked on production after #301 (5355515), with run 144 on Free.** Two of the three points work; point 2 doesn't, so this stays open. - **Point 3, Anthropic's slots: works.** With the Anthropic profile active and no Parallel sessions, `/api/slots` and its history report `capacity: null`, `capacity_source: none`, where they read six before. - **Point 4, the quota figure: works.** The top bar reads `FREE QUOTA all tiers 100% left today`. During the run, beam-free had used 7 of its 200 requests that hour, and every tier still read 100% left: hour and minute limits no longer count. - **Point 2, each step's model on the Inference page: not visible on Free.** The label went into the slot rows, and #289 hides those on hosted backends, so the Free Inference page shows the quota tiers and the Agents chart but no row per running step. Fix: draw the running steps' rows on hosted backends without the slot tiles. Note for #309: a Free step's `serving` carries a name too (`kilo/beam-free`), so "on Free" must come from the profile, not from `serving.name`.
Author
Owner

Point 2 confirmed on production (6f440e5, PR #310); closing.

With Free in use, the Inference page draws a row per running step naming its model (run 145: agent.implement … kilo/beam-free Running commands 4m 22s), where after #301 it drew none. Points 3 and 4 were confirmed with run 144, and point 1 moved to #289.

Not seen live: the waiting for free quota wording for a step held on spent quota, since no step was held. ui/src/lib/free.test.ts covers it.

**Point 2 confirmed on production (6f440e5, PR #310); closing.** With Free in use, the Inference page draws a row per running step naming its model (run 145: `agent.implement … kilo/beam-free Running commands 4m 22s`), where after #301 it drew none. Points 3 and 4 were confirmed with run 144, and point 1 moved to #289. Not seen live: the `waiting for free quota` wording for a step held on spent quota, since no step was held. `ui/src/lib/free.test.ts` covers it.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#199
No description provided.