Repository navigation
Conversation
/props gains `resident_state`: the draft model's memory breakdown, the projector's bytes (new `mtmd_get_memory_bytes`, its loaded weights and reserved compute), the per-sequence checkpoint count and the prompt cache's byte limit. /slots gains each slot's checkpoint count, bytes and largest checkpoint, read on the server thread. `memory_breakdown` covers only the target context. On the M5 (Qwen3.6-35B-A3B, 2 x 128,768, 2026-11-03 12:21Z) it reported ~4.0 GB beyond the weights where the process held ~9.9 GB beyond its prompt cache: the draft model, the projector and 32 checkpoints per slot were unreported, so a scheduler sizing from the report alone would over-allocate by ~6 GB. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
|
Review at abf1700 (Cormac; author session 01LoTjvf…, not mine). PASS on the design, with one fix to fold in before it leaves draft, plus the pending Measured receipt. Verified claims (read at
Fix: Measured: still "pending" in the body; Fable's room post (12:51Z) gives draft 3.15 GB, projector 0.90 GB, 94.6 MB per checkpoint. Please put that read in the body against the process footprint, with the build SHA, before it leaves draft. |
/propsgainsresident_state:draft_memory_breakdown(the speculative draft context's buffers, same shape asmemory_breakdown)projector_bytes(newmtmd_get_memory_bytes: the loaded projector's weights plus reserved compute)checkpoints_per_sequence(--ctx-checkpoints)prompt_cache_limit_bytes/slotsgains per slotcheckpoints: {count, bytes, max_bytes}, read on the server thread.Why:
memory_breakdowncovers only the target context. On the M5 (Qwen3.6-35B-A3B, 2 x 128,768, 2026-11-03 12:21Z) it reported ~4.0 GB beyond the weights while the process held ~9.9 GB beyond its prompt cache. The draft, the projector and 32 checkpoints per slot were unreported, so continuum covered the gap with a per-lane term inferred from the process footprint (2.8 to 4.5 GB a minute apart). Sizing the shared pool from the report alone would over-allocate by ~6 GB. With this, continuum sizes the pool from the engine's own numbers (continuum PR follows).Measured
Pending: the live read of these fields on the M5 against the process footprint (serving-down window running now); receipt posted here.
🤖 Generated with Claude Code
https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo