Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions s08_context_compact/README.ja.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Agent が作業を続けると、読み込んだファイル、コマンド結

コンテキストウィンドウは、モデルが現在使っている下書き用紙と考えられます。ユーザーメッセージ、モデルの応答、`tool_use`、`tool_result` が順番に書き込まれます。モデルはタスクを続けるたびに、その内容を読み直します。

下書き用紙の大きさは固定です。上限を超えると API はリクエストを拒否し、`prompt_too_long` を返します。コーディングタスクでは、ツール結果が多くの領域を占めます。
下書き用紙の大きさは固定です。上限を超えると API はリクエストを拒否し、`prompt is too long` を返します。コーディングタスクでは、ツール結果が多くの領域を占めます。

- 長いファイルを読むと、その内容がコンテキストに入ります。
- テストやビルドのログは、一度に数十 KB 追加されることがあります。
Expand Down Expand Up @@ -202,7 +202,7 @@ if self.estimate_chars(messages) > self.CONTEXT_CHAR_LIMIT:

## API に拒否された後の回復

文字数はモデルが使う token 数の推定値です。そのため API が `prompt_too_long` を返す可能性は残ります。`reactive_compact` は transcript を保存し、古い履歴を要約して、最新 5 メッセージを保持します。
文字数はモデルが使う token 数の推定値です。そのため API が `prompt is too long` を返す可能性は残ります。`reactive_compact` は transcript を保存し、古い履歴を要約して、最新 5 メッセージを保持します。

```python
tail_start = max(0, len(messages) - self.KEEP_RECENT_MESSAGES)
Expand Down Expand Up @@ -236,6 +236,7 @@ def agent_loop(messages, active_request):
except Exception as error:
message = str(error).lower()
too_long = ("prompt_too_long" in message
or "prompt is too long" in message
or "too many tokens" in message)
if too_long and reactive_retries < MAX_REACTIVE_RETRIES:
messages[:] = COMPACTOR.reactive_compact(
Expand Down
5 changes: 3 additions & 2 deletions s08_context_compact/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ This lesson adds a four-step compaction pipeline. It first reduces recoverable t

Think of the context window as the model's current scratchpad. User messages, model responses, `tool_use`, and `tool_result` blocks are written onto it in order. The model reads that material again whenever it continues the task.

The scratchpad has a fixed size. When a request exceeds it, the API rejects the call with `prompt_too_long`. Tool results usually consume most of the space in coding tasks:
The scratchpad has a fixed size. When a request exceeds it, the API rejects the call with `prompt is too long`. Tool results usually consume most of the space in coding tasks:

- Reading a long file puts its contents into the context.
- Test and build logs can add tens of kilobytes at once.
Expand Down Expand Up @@ -202,7 +202,7 @@ Each round therefore starts with the lowest-cost operation whose information is

## Recovering From an API Rejection

A character count can only estimate the tokens used by a model. The API may still return `prompt_too_long`. `reactive_compact` saves a transcript, summarizes older history, and retains the latest 5 messages:
A character count can only estimate the tokens used by a model. The API may still return `prompt is too long`. `reactive_compact` saves a transcript, summarizes older history, and retains the latest 5 messages:

```python
tail_start = max(0, len(messages) - self.KEEP_RECENT_MESSAGES)
Expand Down Expand Up @@ -236,6 +236,7 @@ def agent_loop(messages, active_request):
except Exception as error:
message = str(error).lower()
too_long = ("prompt_too_long" in message
or "prompt is too long" in message
or "too many tokens" in message)
if too_long and reactive_retries < MAX_REACTIVE_RETRIES:
messages[:] = COMPACTOR.reactive_compact(
Expand Down
5 changes: 3 additions & 2 deletions s08_context_compact/README.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Agent 持续工作时,读过的文件、执行过的命令和模型回复都

可以把上下文窗口看作模型当前使用的一张草稿纸。用户消息、模型回复、`tool_use` 和 `tool_result` 都会按顺序写在这张纸上。模型每次继续工作时,都要重新读取这些内容。

草稿纸的大小固定。内容超过上限后,API 会拒绝请求并返回 `prompt_too_long`。在代码任务里,工具结果通常占据最多空间:
草稿纸的大小固定。内容超过上限后,API 会拒绝请求并返回 `prompt is too long`。在代码任务里,工具结果通常占据最多空间:

- 读取一个长文件会把文件内容放进上下文;
- 测试和构建日志可能一次产生几十 KB 文本;
Expand Down Expand Up @@ -202,7 +202,7 @@ if self.estimate_chars(messages) > self.CONTEXT_CHAR_LIMIT:

## API 拒绝后的补救

字符数只能估算模型实际使用的 token。API 仍可能返回 `prompt_too_long`。`reactive_compact` 会保存 transcript,总结较早历史,并保留最近 5 条消息:
字符数只能估算模型实际使用的 token。API 仍可能返回 `prompt is too long`。`reactive_compact` 会保存 transcript,总结较早历史,并保留最近 5 条消息:

```python
tail_start = max(0, len(messages) - self.KEEP_RECENT_MESSAGES)
Expand Down Expand Up @@ -236,6 +236,7 @@ def agent_loop(messages, active_request):
except Exception as error:
message = str(error).lower()
too_long = ("prompt_too_long" in message
or "prompt is too long" in message
or "too many tokens" in message)
if too_long and reactive_retries < MAX_REACTIVE_RETRIES:
messages[:] = COMPACTOR.reactive_compact(
Expand Down
3 changes: 2 additions & 1 deletion s08_context_compact/code.py
Original file line number Diff line number Diff line change
Expand Up @@ -536,7 +536,8 @@ def agent_loop(messages: list, active_request: str):
reactive_retries = 0
except Exception as error:
too_long = any(text in str(error).lower()
for text in ("prompt_too_long", "too many tokens"))
for text in ("prompt_too_long", "prompt is too long",
"too many tokens"))
if too_long and reactive_retries < MAX_REACTIVE_RETRIES:
print("[reactive compact]")
messages[:] = COMPACTOR.reactive_compact(messages, active_request)
Expand Down
71 changes: 71 additions & 0 deletions tests/test_s08_context_compact.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@
import sys
import types
from pathlib import Path
from unittest.mock import Mock

import pytest


ROOT = Path(__file__).resolve().parents[1]
Expand Down Expand Up @@ -136,3 +139,71 @@ def test_prepare_persists_oversized_unseen_result_before_full_compact(
saved_line = next(line for line in content.splitlines()
if line.startswith("Full output: "))
assert Path(saved_line.removeprefix("Full output: ")).read_text() == output


@pytest.mark.parametrize("error_message", [
"prompt is too long: 210445 tokens > 200000 maximum",
"prompt_too_long",
"too many tokens",
])
def test_agent_loop_compacts_and_retries_context_overflow(
tmp_path, monkeypatch, error_message):
lesson = load_lesson(monkeypatch, tmp_path)
messages = [{"role": "user", "content": "continue"}]
compacted = [{"role": "user", "content": "compacted history"}]
requests = []
response = types.SimpleNamespace(
content=[types.SimpleNamespace(type="text", text="Recovered")])

def create(**kwargs):
requests.append(list(kwargs["messages"]))
if len(requests) == 1:
raise RuntimeError(error_message)
return response

compact = Mock(return_value=compacted)
monkeypatch.setattr(lesson["client"].messages, "create", create)
monkeypatch.setattr(lesson["COMPACTOR"], "reactive_compact", compact)

lesson["agent_loop"](messages, "continue")

assert len(requests) == 2
assert requests[1] == compacted
compact.assert_called_once()
assert compact.call_args.args[1] == "continue"
assert messages == [*compacted,
{"role": "assistant", "content": response.content}]


def test_agent_loop_stops_after_one_reactive_retry(tmp_path, monkeypatch):
lesson = load_lesson(monkeypatch, tmp_path)
messages = [{"role": "user", "content": "continue"}]
first_error = RuntimeError("prompt is too long: first request")
retry_error = RuntimeError("prompt is too long: retry")
create = Mock(side_effect=[first_error, retry_error])
compact = Mock(return_value=list(messages))
monkeypatch.setattr(lesson["client"].messages, "create", create)
monkeypatch.setattr(lesson["COMPACTOR"], "reactive_compact", compact)

with pytest.raises(RuntimeError) as caught:
lesson["agent_loop"](messages, "continue")

assert caught.value is retry_error
assert create.call_count == 2
compact.assert_called_once()


def test_agent_loop_propagates_unrelated_errors(tmp_path, monkeypatch):
lesson = load_lesson(monkeypatch, tmp_path)
error = RuntimeError("invalid API key")
create = Mock(side_effect=error)
compact = Mock()
monkeypatch.setattr(lesson["client"].messages, "create", create)
monkeypatch.setattr(lesson["COMPACTOR"], "reactive_compact", compact)

with pytest.raises(RuntimeError) as caught:
lesson["agent_loop"]([{"role": "user", "content": "continue"}], "continue")

assert caught.value is error
create.assert_called_once()
compact.assert_not_called()