bug: Compaction does not work and the chat continues where it left of

[JKGeovision]

Hello,

I discovered that the compaction feature does not work correctly. Eventhough the compaction gets the model to return something the next message continues right where the last one left of. My assumption is that the compaction correctly gets the model to summaries but subsequent request dont cut off the chat history before that.

0177.20.719.593 I srv  proxy_reques: proxying request to model Qwen3.8 27B (medium) on port 55467
[55467] 29.54.311.332 I slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, f_sim_best = 0.996 (> 0.100 thold), f_keep = 1.000
[55467] 29.54.311.928 I slot launch_slot_: id  0 | task 387 | processing task, is_child = 0
[55467] 30.02.697.707 I slot print_timing: id  0 | task 387 | n_gen =    101, tg =  23.37 t/s, tg_3s =  23.60 t/s
[55467] 30.05.736.972 I slot print_timing: id  0 | task 387 | n_gen =    194, tg =  26.37 t/s, tg_3s =  30.60 t/s
[55467] 30.08.774.428 I slot print_timing: id  0 | task 387 | n_gen =    265, tg =  25.49 t/s, tg_3s =  23.37 t/s
[55467] 30.11.814.724 I slot print_timing: id  0 | task 387 | n_gen =    316, tg =  23.51 t/s, tg_3s =  16.77 t/s
[55467] 30.12.301.236 I slot print_timing: id  0 | task 387 | prompt eval time =    4106.47 ms /   640 tokens (    6.42 ms per token,   155.85 tokens per second)
[55467] 30.12.301.238 I slot print_timing: id  0 | task 387 |        eval time =   13882.63 ms /   324 tokens (   42.98 ms per token,    23.27 tokens per second)
[55467] 30.12.301.239 I slot print_timing: id  0 | task 387 |       total time =   17989.10 ms /   964 tokens
[55467] 30.12.301.239 I slot print_timing: id  0 | task 387 |    graphs reused =        383
[55467] 30.12.301.243 I slot print_timing: id  0 | task 387 | draft acceptance = 0.61696 (  211 accepted /   342 generated), mean len =  2.85

// long context i want to compact

[55467] 30.12.304.163 I slot      release: id  0 | task 387 | stop processing: n_tokens = 176122, truncated = 0
10178.12.623.354 I srv  proxy_reques: proxying request to model Qwen3.8 27B (medium) on port 55467
[55467] 30.45.948.746 I slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = 611151107915
[55467] 30.49.168.967 I slot launch_slot_: id  0 | task 504 | processing task, is_child = 0
[55467] 30.53.320.942 I slot print_timing: id  0 | task 504 | prompt processing, n_tokens =   4171, progress = 0.56, t =   3.68 s / 1133.63 tokens per second
[55467] 30.55.275.116 I slot print_timing: id  0 | task 504 | prompt processing, n_tokens =   6219, progress = 0.84, t =   5.61 s / 1107.64 tokens per second
[55467] 30.55.548.147 I slot print_timing: id  0 | task 504 | prompt processing, n_tokens =   6391, progress = 0.86, t =   6.28 s / 1018.43 tokens per second
[55467] 30.56.840.228 I slot print_timing: id  0 | task 504 | prompt processing, n_tokens =   7415, progress = 1.00, t =   7.19 s / 1031.60 tokens per second
[55467] 30.59.959.252 I slot print_timing: id  0 | task 504 | n_gen =    162, tg =  53.53 t/s, tg_3s =  53.86 t/s
[55467] 31.02.962.961 I slot print_timing: id  0 | task 504 | n_gen =    292, tg =  48.41 t/s, tg_3s =  43.28 t/s
[55467] 31.05.972.238 I slot print_timing: id  0 | task 504 | n_gen =    447, tg =  49.44 t/s, tg_3s =  51.51 t/s
[55467] 31.09.000.928 I slot print_timing: id  0 | task 504 | n_gen =    598, tg =  49.55 t/s, tg_3s =  49.86 t/s
[55467] 31.12.049.402 I slot print_timing: id  0 | task 504 | n_gen =    750, tg =  49.61 t/s, tg_3s =  49.86 t/s
[55467] 31.13.253.884 I slot print_timing: id  0 | task 504 | prompt eval time =    7782.71 ms /  7419 tokens (    1.05 ms per token,   953.27 tokens per second)
[55467] 31.13.253.886 I slot print_timing: id  0 | task 504 |        eval time =   16301.98 ms /   794 tokens (   20.56 ms per token,    48.64 tokens per second)
[55467] 31.13.253.886 I slot print_timing: id  0 | task 504 |       total time =   24084.69 ms /  8213 tokens
[55467] 31.13.253.887 I slot print_timing: id  0 | task 504 |    graphs reused =        649
[55467] 31.13.253.891 I slot print_timing: id  0 | task 504 | draft acceptance = 0.64453 (  524 accepted /   813 generated), mean len =  2.93
[55467] 31.13.254.085 I slot      release: id  0 | task 504 | stop processing: n_tokens = 8214, truncated = 0

// returned presumably the summary

10178.39.960.509 E srv    operator(): http client error: Connection handling canceled
10180.54.269.523 I srv  proxy_reques: proxying request to model Qwen3.8 27B (medium) on port 55467
[55467] 33.27.873.571 I slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = 611212057838
[55467] 33.28.965.816 I slot launch_slot_: id  0 | task 783 | processing task, is_child = 0
[55467] 33.33.605.682 I slot print_timing: id  0 | task 783 | n_gen =    103, tg =  26.94 t/s, tg_3s =  27.20 t/s
[55467] 33.36.643.610 I slot print_timing: id  0 | task 783 | n_gen =    197, tg =  28.72 t/s, tg_3s =  30.94 t/s
[55467] 33.39.698.431 I slot print_timing: id  0 | task 783 | n_gen =    285, tg =  28.75 t/s, tg_3s =  28.81 t/s
[55467] 33.40.674.123 I slot print_timing: id  0 | task 783 | prompt eval time =     852.88 ms /    25 tokens (   34.12 ms per token,    29.31 tokens per second)
[55467] 33.40.674.124 I slot print_timing: id  0 | task 783 |        eval time =   10855.21 ms /   316 tokens (   34.46 ms per token,    29.02 tokens per second)
[55467] 33.40.674.125 I slot print_timing: id  0 | task 783 |       total time =   11708.09 ms /   341 tokens
[55467] 33.40.674.125 I slot print_timing: id  0 | task 783 |    graphs reused =        736
[55467] 33.40.674.127 I slot print_timing: id  0 | task 783 | draft acceptance = 0.85019 (  227 accepted /   267 generated), mean len =  3.55

// continues before the compaction?

[55467] 33.40.677.165 I slot      release: id  0 | task 783 | stop processing: n_tokens = 176463, truncated = 0
10181.08.589.202 I srv  proxy_reques: proxying request to model Qwen3.8 27B (medium) on port 55467
[55467] 33.42.179.481 I slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, f_sim_best = 0.998 (> 0.100 thold), f_keep = 1.000
[55467] 33.42.180.025 I slot launch_slot_: id  0 | task 875 | processing task, is_child = 0
[55467] 33.48.601.836 I slot print_timing: id  0 | task 875 | n_gen =    100, tg =  25.31 t/s, tg_3s =  25.56 t/s
[55467] 33.51.663.108 I slot print_timing: id  0 | task 875 | n_gen =    192, tg =  27.39 t/s, tg_3s =  30.05 t/s
[55467] 33.54.717.522 I slot print_timing: id  0 | task 875 | n_gen =    291, tg =  28.92 t/s, tg_3s =  32.41 t/s

Thank you for your time and consideration.

Best regards,
Joshua Krimmer

[owui-terminator]

🔍 Related Issues Found

I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions:

1. 🟢 #162 bug: repeated context compaction can drop every user message and break Qwen/vLLM
   This is directly about automatic context compaction in cptr/computer and describes a failure mode after compaction where the next request is malformed due to how history is retained. The new issue also reports compaction succeeding superficially but the following chat continues with the old context instead of restarting properly.
   by calderanicola

2. 🟢 #209 bug: task silently marked complete when the post-compaction completion returns zero tokens
   This issue is about the request immediately after compaction behaving incorrectly, with compaction followed by a bad continuation/termination outcome. It is closely related to post-compaction request handling, which is the core symptom in the new report.
   by ThalynLabs

3. 🟣 #146 bug: The /compact skill is not working
   Although narrower and closed, it specifically reports that the /compact feature does not work at all, which overlaps with the user's claim that compaction appears ineffective. It may contain the same underlying compaction path or historical fix context.
   by JKGeovision

---

💡 If your issue is a duplicate, please close it and add any additional details to the existing issue instead.

This comment was generated automatically. React with 👍 if helpful, 👎 if not.

[JKGeovision]

Compaction ineffective against single-slot llama.cpp backend: largest conversation grows to context limit while other conversations do compact; internal calls time out and retry-storm the slot

Summary

When cptr runs against a local llama.cpp server (single slot, ~256k context), context compaction does not protect the largest conversation. In a captured llama.cpp log window (≈5 min) one conversation's context grew monotonically from 110,753 → 256,571 tokens (98 % of the model context) without a single compaction, although the default compaction threshold is 80,000 tokens.

At the same time, other, smaller conversations do trigger compaction (see evidence: two summarization requests with the exact max_tokens=1000 signature of summarize_messages()), and several requests from cptr are canceled client-side (http client error: Connection handling canceled, 5× within 31 s) — i.e. from the user's perspective cptr sends multiple concurrent requests, one of them grows until the context limit is hit, and another one times out every time.

The log also shows the systemic consequence: 4–5 concurrent conversations competing for one slot cause prompt-cache thrashing (a single eviction of 15,366 MiB of KV cache) and full re-evaluation of 100k+ token prompts (215 s, 117 s, 114 s, 116 s), which in turn makes every queued internal call (summarize/review/approval) exceed its hard 15 s/20 s client timeouts.

This is cptr used natively (not via the OpenAI-compatible gateway) with llama.cpp configured as an OpenAI-compatible connection.

Environment

• cptr 0.9.21 (pip wheel)
• llama.cpp llama-server, single slot (id 0), model "Qwen3.8 27B", context 262,144 tokens, speculative decoding enabled
• Frontend llama-server in proxy mode proxying to a backend llama-server on port 55467 (log lines srv proxy_request: proxying request to model ... on port 55467)
• One user, multiple cptr chats active concurrently (main chat + at least two further conversations/sub-agents)
• Default CHAT_COMPACT_TOKEN_THRESHOLD (80,000) assumed — please confirm the effective threshold in this setup (see "Diagnostics requested")

Reproduction

1. Configure cptr with a single-slot local llama.cpp backend (proxy mode optional).
2. Open multiple chats (or use delegate_task sub-agents) so that several cptr conversations issue requests against the same slot.
3. In one conversation, accumulate token-dense context (tool output, code, log text) past 80k tokens, e.g. by analyzing large files/logs.
4. Keep other conversations active so the slot is busy.

Observed: the large conversation is never compacted and keeps growing until the model context limit; internal LLM calls (summarization, reviews) time out; llama.cpp logs show client-side connection cancellations and massive KV-cache evictions.

Evidence from the llama.cpp log

1. The growing conversation is never compacted (110k → 256k, threshold 80k)

All requests of this conversation are selected by LCP prefix similarity (0.959–1.000), i.e. one conversation:

Task	New prompt tokens	Generated	Context at end (n_tokens)	Total wall time
59335	109,772 (full re-eval, 215.6 s)	981	110,753	247.2 s
59999	1,310	709	112,770	30.8 s
60263	4,810	715	118,294	41.8 s
60524	2,408	441	121,143	22.4 s
60656	1,905	664	123,713	31.5 s
60888	384	458	124,554	18.4 s
61052	969	375	125,899	19.6 s
61201	735	2,020	128,653	78.2 s
61929	22,768 (re-eval 117.5 s)	2,320	224,398	227.4 s
66333	14,244 (re-eval 81.6 s)	108	238,750	86.7 s
67243	2,380	877	242,007	67.4 s
69995	9,621 (re-eval 58.9 s)	4,942	256,571	306.4 s

Already at the start of the window the conversation is 30 %+ above the 80k threshold, and it never drops — every iteration only appends (a few hundred to a few thousand new tokens). No summarization request attributable to this conversation appears anywhere in the window.

Raw excerpts:

[55467] 447.16.730.825 I slot print_timing: id 0 | task 59335 | prompt eval time = 215553.42 ms / 109772 tokens
[55467] 447.16.732.618 I slot      release: id 0 | task 59335 | stop processing: n_tokens = 110752, truncated = 0
...
[55467] 458.52.948.409 I slot      release: id 0 | task 61929 | stop processing: n_tokens = 224398, truncated = 0
...
[55467] 485.03.268.616 I slot      release: id 0 | task 69995 | stop processing: n_tokens = 256571, truncated = 0

2. Other conversations DO compact — but only by the exact summarize_messages() signature

Two independent requests in the window have the unmistakable signature of cptr's compaction summarization call: prompt of ~7.3–7.8k tokens (system prompt + the ≤30,000-char message dump built in summarize_messages) and exactly 1,000 generated tokens (the max_tokens=1000 cap passed to generate_text — see utils/summarize.py:75):

[55467] 459.05.838.178 I slot print_timing: id 0 | task 62723 | prompt processing, n_tokens = 7755, progress = 1.00
[55467] 459.24.996.191 I slot print_timing: id 0 | task 62723 | eval time = 19045.96 ms / 1000 tokens   <-- hit max_tokens=1000 exactly
[55467] 459.24.996.390 I slot      release: id 0 | task 62723 | stop processing: n_tokens = 8833, truncated = 0
...
[55467] 485.15.083.138 I slot print_timing: id 0 | task 71618 | prompt processing, n_tokens = 7314, progress = 1.00
[55467] 485.35.688.959 I slot print_timing: id 0 | task 71618 | eval time = 19151.18 ms / 1000 tokens   <-- hit max_tokens=1000 exactly
[55467] 485.35.689.167 I slot      release: id 0 | task 71618 | stop processing: n_tokens = 8392, truncated = 0

So the compaction pipeline works in principle — but the largest conversation (the one most in need of compaction) is precisely the one it never fires for. The most plausible mechanism is the estimation fallback in should_compact() (see Root cause 1): token-dense content underestimates real token counts by ~2×, so a real-110k-token dense conversation can still look like ~55–75k to the decision, while lighter (more natural-language) conversations of 80–90k tokens do cross the threshold.

3. Client-side cancellations ("the request that always times out")

Five client disconnects are logged by the (front) server within ~31 s, each while requests of other conversations were still running/finishing:

10606.51.702.837 E srv operator(): http client error: Connection handling canceled
10618.41.402.002 E srv operator(): http client error: Connection handling canceled
10621.24.640.492 E srv operator(): http client error: Connection handling canceled
10633.02.395.459 E srv operator(): http client error: Connection handling canceled
10637.53.972.876 E srv operator(): http client error: Connection handling canceled

4. Concurrent conversations + KV-cache thrashing

Besides the growing conversation, at least three further independent conversations are active in the same window (different prefix chains):

• Conversation B: tasks 62723 → 66147, context 8,833 → 84,580 (incl. one +71,262-token jump, task 62770, 210 s total)
• Conversation C: tasks 66379 → 69292, context 8,613 → 97,549 (incl. +71k jump pattern)
• Conversation D: tasks 71618 → 73434, context 8,392 → 77,795 (incl. +71,865-token prompt, task 71671, 128 s total)
• Conversation E: task 73575, context 3,376

Because only one slot exists, these serialize, and the prompt cache cannot hold them all:

[55467] 455.02.015.933 W srv alloc: - making room for prompt cache entry, removing oldest entry (size = 15366.401 MiB)
[55467] 479.54.031.436 W srv alloc: - making room for prompt cache entry, removing oldest entry (size = 936.803 MiB)
[55467] 485.03.390.205 W srv alloc: - making room for prompt cache entry, removing oldest entry (size = 694.618 MiB)
[55467] 490.11.082.796 W srv alloc: - making room for prompt cache entry, removing oldest entry (size = 7100.080 MiB)

Result: the next request of an evicted conversation pays a full re-evaluation at ~200 tokens/s (215.6 s for 109,772 tokens; 117.5 s for 22,768; 114.0 s for 71,262; 116.7 s for 71,865). While one such monster request occupies the slot, every other in-flight cptr request sits in the queue — which is exactly when cptr's short internal-call deadlines (15 s/20 s) and stream read timeouts fire.

Root cause analysis (cptr 0.9.21 source)

RC1 (primary hypothesis) — Compaction decision degrades to a len/4 estimate when provider usage is absent/zero, which underestimates token-dense content

utils/context.py:33-60 (should_compact):

last_usage = normalize_usage(last_usage)
if last_usage and last_usage.get("input_tokens"):
    base = last_usage["input_tokens"] + last_usage.get("output_tokens", 0)
    ...
    return base > resolved_threshold

# Full estimation fallback
total = estimate_tokens(system_prompt) + estimate_messages_tokens(messages)
return total > resolved_threshold

with estimate_tokens(text) = max(1, len(text) // 4) (utils/context.py:8-10).

Two problems:

1. When does the fallback trigger? Whenever the streaming response delivers no (or zero) usage. cptr itself already knows this can happen for llama.cpp: utils/chat_task.py:2255-2283 and 2347-2356 carry a dedicated provider_type == "llama.cpp" and usage_context_tokens(last_usage) <= 0 branch that estimates context usage for the UI meter — but the compaction decision in the same loop does not get any equivalent protection; it silently runs on the len/4 estimate.
2. The estimate is systematically too low for agent workloads. Tool output, JSON, code, logs, and command output routinely tokenize at 2–3 characters per token (not 4). A real 110k-token dense conversation can therefore be estimated at 55–75k tokens — below the 80k default threshold — and never compacts, while a lighter 85k-token natural-language conversation estimates to ~85k and does compact. This exactly matches the log asymmetry in Evidence 1 vs 2 (the largest conversation never compacts; two smaller ones do).

Note: llama.cpp only emits usage in the final stream chunk when stream_options: {"include_usage": true} is set (cptr does send it — utils/ai.py:652), but any intermediary in the chain (llama-server proxy mode, older server builds, reverse proxies) can drop or rewrite that final chunk. Whether usage actually reaches cptr in this setup is the single most important thing to confirm (see Diagnostics).

RC2 — Keep-zone is sized by message count, not tokens → no convergence guarantee

utils/chat_task.py:2135-2138:

target_keep = max(2, len(messages) * 2 // 5)      # keep newest 40% by count
split_idx = _find_safe_split(messages, target_keep)
drop_zone = messages[:split_idx]
keep_zone = messages[split_idx:]

A recent single message can dominate the whole context: tool results are capped at 50,000 characters (CHAT_TOOL_MAX_CHARS, env.py:63 ≈ 12.5–25k tokens depending on density), run_command output at 8,000, user-pasted content is unbounded, and images are worth ~1k tokens each. If the newest 40 % contains such a message, compaction drops mostly small old messages, the context stays above the threshold, and should_compact() fires again on the very next iteration — an LLM summarization call per iteration that never converges. There is no "after compaction we are still above threshold → also cap/truncate keep-zone messages" logic.

RC3 — The summarization call has hard 15 s / 20 s deadlines that a busy local backend cannot meet → silent degradation to a 2,000-char stub

utils/summarize.py:65-83 calls:

result = await generate_text(
    model_id=await Config.get("chat.context_compaction.model"),
    active_connection=connection,
    active_model=model,
    messages=[{"role": "user", "content": text}],
    system=_get_summarize_prompt(),
    max_tokens=1000,
)

• generate_text wraps the call in asyncio.wait_for(..., timeout=20) (utils/ai.py:244, 287-300),
• and the underlying chat_completion uses httpx.AsyncClient(timeout=httpx.Timeout(15)) — a hardcoded 15 s connect/read/write timeout (utils/ai.py:136).

On a single-slot local backend, the summarization request first waits in the queue behind whatever long request is running (in the log: 30–300 s), then needs its own prompt eval. 15 s/20 s is not even a fraction of that. Every such call therefore:

1. times out on the client side → the backend logs Connection handling canceled (Evidence 3), and
2. falls back to _naive_summary (utils/summarize.py:86-96): a 2,000-character stub of ≤200-char snippets per user/assistant message, which is then persisted as the conversation checkpoint (ChatMessage.update(checkpoint_message_id, chat_summary=summary), utils/chat_task.py:2150). Each failed pass overwrites the previous summary with a worse one — compaction silently degrades the conversation instead of protecting it.

RC4 — Even on success, the summary is hard-capped at 1,000 tokens

utils/summarize.py:75 passes max_tokens=1000. Both summarization requests in the log hit the cap exactly (eval time ... / 1000 tokens), i.e. the summary was truncated mid-sentence. Combined with the input caps (≤2,000 chars per message, ≤30,000 chars total, utils/summarize.py:56-63), a drop zone of 100k+ tokens compresses to a 1,000-token stub.

RC5 — No concurrency control for the many internal LLM calls against the same single slot

For a single user turn, cptr can issue, in addition to the main streaming request:

• tool-approval review per reviewed tool call — utils/chat_task.py:739-796 (generate_json, 20 s deadline)
• background memory review — detached via asyncio.create_task at end of turn (utils/chat_task.py:2842-2857 → utils/memory.py:1574-1583), so it overlaps the next user turn
• background skill review — detached the same way (utils/chat_task.py:2858-2877 → utils/skills.py:776-781)
• chat title generation (utils/chat_task.py:799-846)
• the summarization call of RC3 (20 s deadline)

…all against the same single-slot backend, plus every other open chat or delegate_task sub-agent. There is no queue, semaphore, or priority between the interactive main request and these internal calls. In the log this manifests as the multiple concurrent conversations and the burst of client-side cancellations.

RC6 — Stream retries re-send the entire request up to 3× (thundering herd on one slot)

utils/ai.py:56-58, 816-829: on any retryable stream error before the first event — including httpx.ReadTimeout, which is what a client-side timeout looks like when the first attempt is still queued/processing on the backend — the full request body (i.e. the whole, potentially 100k+ token context) is re-sent, up to 3 times with only 0.5–1.5 s backoff. On a single slot this adds full-context re-evaluations to an already saturated queue instead of recovering.

Expected behavior

• A conversation crossing the configured compact_token_threshold should be compacted before it approaches the model context limit, and compaction should converge (post-compaction context below threshold).
• Compaction decisions should be based on real token counts; when provider usage is unavailable, the fallback estimate must not systematically undercount the exact workloads (tool output, code, logs) that make agent conversations grow.
• Internal LLM calls (summarize, reviews, approvals, titles) must use backend-appropriate timeouts and must not silently replace good summaries with naive stubs.
• Multiple cptr conversations/turns against a single-slot backend should not produce retry storms or client-side cancellations.

Proposed fixes

1. Make the compaction decision robust (RC1):
   • Log the decision (threshold, base tokens, source: usage vs estimate) at INFO level so "why didn't it compact" is answerable from cptr logs.
   • When falling back to estimation, apply a safety factor (e.g. ×1.3–1.5) or use a density-aware estimate, so token-dense content cannot hide below the threshold.
   • Optionally: for provider_type == "llama.cpp", treat "no usage in stream" as a known degraded mode and lower the effective threshold (or surface a warning) rather than silently estimating.
2. Converge compaction (RC2): keep a token-budgeted tail (e.g. newest messages up to ~50 % of the threshold) instead of 40 % by count; if the keep-zone still exceeds the threshold, also cap oversized individual messages in the model view (truncate very old tool results / spill to file with a pointer) or keep dropping until below threshold (with a minimum-message floor).
3. Backend-appropriate summarization (RC3):
   • Make the summarization call streaming (or at minimum use a configurable, much larger timeout for local backends; expose e.g. chat.context_compaction.timeout_seconds in config, default ≥ 600 s for non-cloud providers).
   • On timeout/failure, do not overwrite an existing checkpoint summary with the naive stub — keep the old summary and retry on a later compaction, and raise the failure to WARNING with context size included.
4. Higher summary budget (RC4): raise max_tokens for summarization (e.g. 2,000–4,000) or make it proportional to the drop-zone size, so the checkpoint summary is not truncated mid-sentence.
5. Serialize internal calls (RC5): run non-interactive internal calls (memory/skill review, approval, title, summarize) through a single-flight queue per connection, and let the interactive main stream have priority; consider deferring background reviews when the backend is known to be single-slot/local.
6. Saner retry policy (RC6): do not blindly re-send a full-context request after a client-side read timeout (the original attempt may still be processing upstream); make attempt count/backoff configurable and consider cancel-then-wait for the original before retrying.

Diagnostics requested (to confirm RC1 vs. alternatives)

1. Effective threshold: value of chat.compact_token_threshold in admin UI app_config / [chat] compact_token_threshold in config.toml / CHAT_COMPACT_TOKEN_THRESHOLD env (expected: 80000 default).
2. cptr log around the affected time, CPTR_LOG_LEVEL=DEBUG, especially:
   • lines matching compacted: (are there any for the growing conversation?),
   • [summarize] LLM call failed, using naive fallback (RC3 occurrences),
   • [task …] db save / iteration counts.
3. Usage reachability check (isolates RC1): against the front (proxy) server, run a raw streaming request with "stream_options": {"include_usage": true} and check whether the final chunk contains usage:

   curl -N http://<front-server>/v1/chat/completions -H 'Content-Type: application/json' -d '{
     "model": "Qwen3.8 27B",
     "messages": [{"role": "user", "content": "hi"}],
     "stream": true,
     "stream_options": {"include_usage": true}
   }' | tail -5

   If the final chunk has no usage (or prompt_tokens: 0), the compaction decision runs on the len/4 estimate — confirming RC1.
4. llama.cpp versions of both the front (proxy) and backend server, plus exact server launch flags (slots, cache size, --no-warmup, etc.).
5. Which conversations were active at the time (number of open chats, any delegate_task sub-agents) to map log tasks 62723/71618 to their parent conversations.

Workarounds (until fixed)

• Lower compact_token_threshold well below the model context (e.g. 40–60k) and keep conversations with dense tool output shorter.
• Run llama.cpp with more slots if VRAM allows, or raise the prompt-cache budget, to reduce queue times so the 15 s/20 s internal-call deadlines are occasionally met.
• Avoid multiple concurrent cptr chats against one local model; let turns finish before starting the next.