The slot strip reads 1/6 on a llama.cpp server that serves one slot: the total is a constant, not the server's #184
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Reported by the user on 2026-10-17. While trogdor ran llama.cpp at
-np 1(run 67), the top bar's slot strip showed 1/6, though the server serves one sequence at a time.Cause
/api/slotsand/api/slots/historyboth reportcapacityfromConfig.max_concurrent_upstream. That isMAX_CONCURRENT_UPSTREAM = 6, measured once for vLLM's--max-num-seqs 6(§0a) and overridable only by the environment,OSF_MAX_UPSTREAM. Nothing asks the server.Wanted
total_slotsfrom itsGET /props. Read it at most once a minute, since tabs poll the strip.--max-num-seqs, so it keeps the configured value, as does a server that does not answer./api/slotssays where its number came from, so the hover can say "reported by llama.cpp" or "configured".total_slots: 1: the strip reads 1/1 and the Inference page draws one row.Shipped in
0a4d053(changeslots-are-the-servers, commitb3a451d), live since 03:53Z on 2026-10-17, and archived incfb2a50.total_slotsfrom itsGET /props(the-npit was started with), read at most once a minute.--max-num-seqs, so it keeps the configured count. So does a server that doesn't answer./api/slotsand/api/slots/historysay which (capacity_source), and the strip's sentence says "the count the server reports" or "Braid's configured count"./api/slotsreads 6,configured. trogdor runs vLLM now, so the llama.cpp path stands on a unit test with a fake/propsansweringtotal_slots: 1(the strip reads 1/1 and draws one slot). The next llama.cpp profile will show it live.Closing. Please reopen if a llama.cpp server's strip still reads 6.