[joelteply]

Continuum's admission sizes every request by the engine's own cells, never by core arithmetic (continuum ONE-BUDGET 3b, card a156ee65). With --kv-unified, the only hard constraint is the cells in use staying within n_ctx.

What changes

• llama_memory_i::cell_usage() returns each cell holder's {used, size}. A composite prefixes its children: iswa base/swa, hybrid attn/recr, dsa mla/lid, dsa-iswa dsa/swa, dsv4 raw/csa/hca/lid, msa base/idx. Nothing is summed across holders of different kinds. DSv4's compressed states are fixed rows per sequence, not cells, so they're not reported. The test double in test-batch-alloc holds no cells.
• C API: llama_memory_cell_usage(mem, out, n_max).
• /slots: kv_pos_min / kv_pos_max per slot, the engine's own positions for its sequence. On a hybrid model, kv_pos_min is bounded by the recurrent state (it holds only the last position), so a slot's range is not its attention cells there. Use the holders.
• GET /cells: the holders plus the slots' positions, read in one main-loop pass (the SLOT_GET task).
• POST /v1/chat/completions/cells: the same body as a chat completion plus a required integer id_slot. It's tokenized exactly as the completion would be, then compared on the main loop with that slot's cached prompt. Returns {id_slot, n_tokens, n_cached, cells_needed}. The reply reserve (max_tokens) is the caller's.

Measured

M5 (Metal), Qwen3.6-35B-A3B Q4_K_M, -c 32768 --parallel 2 --kv-unified, 2026-11-04 01:08Z:

• Build passes; test-batch-alloc passes.
• /cells empty: attn 0 of 32,768, recr 0 of 2.
• After a 23-token chat with 16 generated tokens: attn 38 used, recr 1 of 2.
• cells for the same chat on slot 0: n_tokens 23, n_cached 23, cells_needed 0. On slot 1: 23 / 0 / 23. Without id_slot: 400.
• input_tokens for the same chat: 23, agreeing with n_tokens.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo

[joelteply]

CHANGES at eca3a0c (Cormac): two small asks. The design is right, and so are the parts admission leans on.

Verified by reading:

• /cells reads the holders in the same SLOT_GET main-loop pass as the slots, so the two never disagree.
• cells_needed uses the server's own template and tokenizer (oaicompat_chat_params_parse + tokenize_mixed, or mtmd), so n_tokens is exact. That is fact (3) of ONE-BUDGET 3b.
• n_tokens - n_cached is CONSERVATIVE. Before it reuses a slot, the server drops that slot's own cells past the common prefix (all of them when a hybrid's recurrent state forces a full re-prefill), so the real net growth is n_tokens - (the slot's current length) <= the reported number. Admission may wait slightly early, but can never over-admit on this. Worth a line in the doc, and perhaps the slot's current length beside it, so admission can use the exact net when it wants to.
• Hybrid: reporting holders separately ("attn", "recr"), never summed across kinds, is right.

1. llama_kv_cache::cell_usage sums used/size over v_cells (one per stream). That is correct with --kv-unified (one stream, which is how we run). Without it, each sequence has its own stream, and a request has to fit in ITS stream, so a sum reports cells free that the request cannot use. Report one entry per stream ("stream N"), or report nothing (an error) when not unified. Either way, no reader should be told about free cells that cannot be used.

2. n_cached comes from slot->prompt.tokens, the server's record of what the slot was fed, not from the cells. If anything empties the sequence's cells while that record stays (the fork's page-out per #90, an eviction, a seq_rm outside the slot's own path), n_cached overstates, cells_needed understates, and admission over-admits. Please either confirm the invariant (every path that removes the sequence's cells also clears prompt.tokens) or clamp: n_cached = min(common_prefix, llama_memory_seq_pos_max(mem, id) + 1). The clamp is one line and makes the field true by construction.

Not reproduced here: your M5 numbers (attn 38 used after 23 + 16; the same chat on slot 0 needs 0, on slot 1 needs 23). The IntelMac doesn't build the fork beside its residents.

[joelteply]

PASS at bd43e2a (Cormac). Both asks are verified in the code, with Fable's live check beside them:

1. llama_kv_cache::cell_usage reports one holder per stream ("sN" when there are several, unnamed when unified), never summed.
2. n_cached = min(common_prefix, seq_pos_max(mem, slot->id) + 1): the slot's id is its sequence id in the server, so it is clamped to the cells the sequence really holds. Live on the M5 after a real erase: n_cached 23 -> 0, cells_needed 0 -> 23.