achildress83 · Oct 19, 2026 · Oct 18, 2026
diff --git a/BACKLOG.md b/BACKLOG.md
@@ -53,225 +53,22 @@ breaking release. Either do it now, or record that it waits.
 
 ## High priority
 
-### 7. A truncated tool call can repeat at full cost until the iteration limit
-
-**Status:** bug. `loop.py` `_Run.drive`, after a turn ends with `_is_truncated_mid_call`:
-
-```python
-self.max_tokens = min(self.max_tokens * 2, MAX_TOKENS_CEILING)
-```
-
-- **At the ceiling, nothing changes and it retries anyway.** Suppose a model keeps writing
-  a tool call larger than 64,000 output tokens (a big file write). Every iteration then
-  discards a full 64k-token turn and asks again with the same budget, up to
-  `max_iterations` (default 50). That is up to 3.2M billed output tokens for an answer
-  that can never fit.
-- **Above the ceiling, the budget shrinks.** A caller who passes `max_tokens=100_000` is
-  cut to 64,000 on the first truncation. That makes the next truncation more likely.
-
-**What implementing it takes:**
-- Never lower a budget: `max(self.max_tokens, min(self.max_tokens * 2, CEILING))`.
-- When a truncation happens at a budget that can no longer grow, stop instead of retrying
-  blind. One option is a non-retryable error, or a dedicated `ConversationStatus`, that
-  says "the tool call does not fit in max_tokens".
-- Add a test with `FakeClient` that scripts repeated truncations at the ceiling.
-
-### 8. An empty turn is retried as if it were transient
-
-**Status:** design gap. `ModelResponse.content` requires `min_length=1`, so an empty
-assistant turn becomes `ResponseInvalid`. That error is `retryable = True`, so
-`RetryingClient` re-sends the identical request.
-
-Two real cases where a retry is wasted money:
-
-- **Completions, `finish_reason: "length"` with empty content.** A reasoning model spends
-  the whole budget on hidden reasoning. With no readable reasoning there is no
-  `ThinkingPart`, so the turn is empty. It is retried `max_attempts` times at the same
-  `max_tokens`, every attempt billed, and then the conversation fails. The loop's
-  `max_tokens` doubling (item 7) never runs, because the loop never sees a response.
-- **Anthropic, an empty `end_turn` right after tool results.** This is a known Claude
-  behaviour, and re-sending the same request does not reliably change it.
-
-**What implementing it takes:** treat "empty and truncated" as truncation, not
-invalidity. Give the loop a way to see it, either a distinct non-retryable error or
-`ModelResponse` allowing empty content when `stop_reason` is `MAX_TOKENS`, so it can raise
-the budget or report `TRUNCATED`. Decide separately whether an empty `END_TURN` should
-end the conversation with empty text or be retried once with a nudge.
-
-### 9. Tool names the model made up reach summaries, metrics and span attribute keys
-
-**Status:** breaks the "content-free" and "low-cardinality metrics" invariants (CLAUDE.md).
-`SummaryCollector.on_tool_end`, `RepeatTracker`, `MetricsObserver` (`tool` attribute) and
-`OTelObserver.on_conversation_end` (`agentloop.tool.calls.{name}` attribute **keys**) all
-take the name from `call.tool_name`. Those names are whatever the model wrote, including
-calls to tools that are not registered. `_execute` answers those with "Unknown tool", but
-the name has already been counted.
-
-**Concrete trigger:** a model invents a new tool name each run (common with small
-open-weight models on Modal or OpenRouter). Each name becomes a new metric series and a
-new span attribute key, and model-written text ends up in a "content-free" summary.
-
-**What implementing it takes:** in `_run_one` / the collector, report unregistered names
-under one fixed bucket such as `"<unregistered>"`. The executor still answers the model
-with the real name. Apply the same mapping in `RepeatTracker`'s reported name, the metrics
-and the root span. Add a test asserting that a hallucinated name never appears in a
-summary or metric.
-
-### 10. No timeout on individual tool-handler execution
-
-**Status:** not implemented. `run_conversation`'s `deadline_seconds` is checked only
-*between* model calls (`loop.py`). Neither `tooling.run_tools`'s single-call path nor its
-`ThreadPoolExecutor` path (`future.result()`, no `timeout=`) limits how long a tool
-handler may run.
-
-**Concrete trigger:** register a tool whose handler makes a network call with no timeout of
-its own, to an endpoint that never responds. The whole agent loop hangs forever, even with
-`deadline_seconds` set. The deadline only fires before the *next* model call, and that
-call never comes.
-
-**What implementing it takes:** a per-tool or per-call timeout, probably a `ToolSpec` field
-that `run_tools` passes to `future.result(timeout=...)`, with an equivalent guard on the
-single-call path. It needs a decision on what happens to a handler thread still running
-after its timeout fires: Python cannot kill a thread.
-
-### 11. `OTelObserver` has no `redact`, unlike `EventTraceObserver`
-
-**Status:** not implemented. `EventTraceObserver(capture_content=True)` refuses to start
-without a `redact` function, or an explicit `allow_unredacted=True`, and redacts every
-captured string before clipping it. `OTelObserver(capture_content=True)` takes no `redact`.
-System instructions, tool arguments and results, and error and retry messages go to the
-exporter as written. `ERROR_DETAIL_CHARS` (200) limits their size, not their content.
-`Agent(trace=True)` cannot enable capture at all, since `_with_tracing` does not pass
-`capture_content` through.
-
-**Concrete trigger:** capture is on to debug a tool, and a tool result or a provider's 400
-body quotes an API key or a customer record. The event trace would redact it; the span
-exports it.
-
-**What implementing it takes:**
-- Add `redact: Callable[[str], str] | None` and `allow_unredacted` to `OTelObserver`, with
-  the event trace's rules: capture without either raises `ConfigurationError`; redact
-  before clipping; a redactor that raises drops the attribute instead of writing it.
-- Decide whether `Agent(trace=...)` should accept capture options or keep capture off by
-  design.
-- Optionally move the redaction helpers in `events.py` to a shared module, so the two
-  observers cannot drift.
-
-### 12. Documentation still describing 0.1 behaviour (hosted tools, Responses, `Origin`)
-
-**Status:** partly done. Already written: the `docs/api/agentloop.md` exports,
-`docs/api/config.md` / `getting-started/configuration.md` for `OPENAI_API`, and the
-`provider.md` coverage of `PAUSED` / `HostedTool` / `hosted_tools` / `origin`. Also
-`transcript.md`'s `HostedToolPart` / `citations` / `Source` / `Citation` /
-`Message.origin`, and the agentloop-tools README.
-
-**Stale pages (wrong today):**
-
-- `docs/api/transcript.md`:
-  - `final_text` is still described as "Concatenates". It now uses `join_text`, which
-    puts a space between separate blocks, but not across Anthropic's citation splits or
-    before closing punctuation.
-  - Missing: `add_assistant_message(parts, origin=None)`, `extend_last_assistant`, and the
-    `check_invariants` rule that hosted parts appear only in assistant messages.
-- `docs/api/provider.md`: `supported_hosted_tools(client)`, and the optional
-  `hosted_tools` attribute a client may expose.
-- `docs/api/agent.md`: a `ToolSpec` named like a hosted tool raises `ConfigurationError`.
-- `docs/api/loop.md`: the `PAUSED` continuation. It sends no user message, joins the same
-  assistant turn, and counts as an iteration.
-- `docs/api/providers/openrouter.md`, `modal.md`: `api="completions"` and empty
-  `hosted_tools` for both. OpenRouter replays `reasoning_details` from `Origin`.
-- `docs/api/providers/anthropic.md`:
-  - `HOSTED_TOOLS` and `API = "messages"`.
-  - `WEB_SEARCH_BASIC` (`web_search_20250305`) is the default. `WEB_SEARCH_FILTERED`
-    (`web_search_20260209`) is sent only when `dynamic_filtering=True`.
-  - `hosted_tool_to_wire`.
-  - `pause_turn` maps to `PAUSED`.
-  - Assistant turns are replayed from `origin.raw`.
-- `docs/api/tracing.md`: the `agentloop.hosted_tool.calls` span attribute (a count only).
-- `docs/api/errors.md`, `docs/guides/retries-and-errors.md`: an OpenAI 429 whose
-  `type`/`code` is `insufficient_quota` / `credit_balance_exhausted` is
-  `InsufficientCredits` (not retried).
-- `docs/guides/custom-providers.md`: `StopReason.PAUSED`. Also how a custom client opts in
-  to hosted tools (the `hosted_tools` attribute) and to exact replay
-  (`ModelResponse.origin`).
-
-**New content needed:**
-
-- `docs/guides/providers.md`: Chat Completions vs Responses.
-  - What differs, and which providers speak which.
-  - Why Responses runs **stateless**: `store: false`, the full transcript resent, item ids
-    stripped on replay.
-  - How to switch: `OPENAI_API`.
-- `docs/guides/conversations.md` / `transcript-storage.md`: what `Message.origin` is, why
-  it can be large (encrypted search results), and that JSONL keeps it so a resumed
-  conversation replays exactly.
-- A **hosted tools / web search** guide:
-  - `native_web_search(...)` from agentloop-tools: its options and each provider's handling
-    (OpenAI `search_context_size`; Anthropic `dynamic_filtering`, below).
-  - The option policy: unknown and restriction options raise; tuning options are logged.
-  - Approval is per request, not per search.
-  - Native and client-side search are separate choices.
-  - **Sources vs citations: the two evidence readers mean different things.** Say so
-    plainly, with this table:
-
-    | | `sources(transcript)` | `citations(transcript)` |
-    |---|---|---|
-    | What it is | Every page the provider's search returned (`Source(url, title)`), deduplicated | A claim in the model's text tied to a page (`Citation`) |
-    | Comes from | `HostedToolPart.sources` (search result blocks / `web_search_call.action.sources`) | `TextPart.citations` |
-    | Anthropic, basic search (default) | Filled | Filled, with `cited_text` (the quoted source text) |
-    | Anthropic, `dynamic_filtering=True` | Filled | **Always empty** (see below) |
-    | OpenAI Responses | Filled (needs the `include` agentloop sends) | Filled, with `start`/`end` (a span of the answer), no `cited_text` |
-    | Use it for | "What did it look at?" A reading list, an audit trail | "Where did this sentence come from?" Footnotes, fact-checking |
-
-    A source is not evidence that the answer used the page; a citation is. Neither one is
-    readable page text: Anthropic encrypts result content, and OpenAI returns only URLs.
-  - **`dynamic_filtering` (Anthropic only; default `False`).**
-    - *What it does:* `True` sends `web_search_20260209`. Claude writes and runs a Python
-      program, in an auto-provisioned code-execution sandbox, that calls the search,
-      filters the results and prints what it keeps. Claude reads only that output.
-      `False` sends `web_search_20250305`, and results go straight into the model's
-      context.
-    - *Why citations disappear:* a citation must point at a search-result block in the
-      model's context, and with filtering the results go to the program instead.
-      Measured live in September 2026 (`claude-opus-5`, 2 questions × 3 variants):
-      filtering on gave 0 citations every time; basic search gave 2–3;
-      `web_search_20260209` with `allowed_callers: ["direct"]` gave 2–5. So the cause is
-      the code path, not the tool version.
-    - *Examples to show:* one response of each. Use the frozen captures in
-      `agentloop-tools/experiments/examples/citations/`, for example
-      `shannon_A_dynamic.json` next to `shannon_B_basic.json`. `experiments/` is
-      gitignored, so on a fresh clone rerun `experiments/citations_experiment.py` first
-      (6 requests, a few cents).
-    - *Cost:* it did NOT save tokens on single simple searches (13,300 vs 12,827 input
-      tokens). Recommend it only for search-heavy requests that don't need citations.
-    - *Constraints:* Claude 4.6+ only; not ZDR-eligible; the response carries extra
-      `server_tool_use` / `code_execution_tool_result` blocks, surfaced as a
-      `HostedToolPart` of kind `code_execution`.
-    - *OpenAI:* has no equivalent. `dynamic_filtering=True` logs a warning and is ignored.
-  - **Domain policy (`agentloop_tools.web.DomainPolicy`).** It prevents and reports, and
-    deliberately does nothing else.
-    - *Modes:* `DomainPolicy.block([...])` or `DomainPolicy.allow([...])`, not both.
-    - *Matching:* a rule covers its host and every subdomain, but not a longer label
-      (`europa.eu` covers `eur-lex.europa.eu`, not `noteuropa.eu`). A rule may add a path
-      prefix in which `*` matches within the path.
-    - *Prevent:* `policy.native_web_search(**options)` sends the list to the provider.
-      Passing a domain option there as well is an error.
-    - *Report:* `policy.audit(transcript, since=n)` returns `Violation(url, where,
-      message_index, rule)`. `where` is `"source"`, `"citation"` or `"text"`. To audit one
-      turn, pass the `len(transcript.messages)` taken before `converse`.
-    - *Text scanning:* full URLs in both modes, with balanced parentheses kept. Bare host
-      mentions are checked in block mode only.
-    - *The hard limit:* with native search, no library can stop the model READING a page.
-      The policy prevents what the provider can and reports the rest. Only client-side
-      search (item 22) filters before the model sees anything.
-    - *Live evidence (September 2026):*
-      - `block(["wikipedia.org"])` held on both providers (0 leaked sources out of 10
-        and 31).
-      - Asked for Wikipedia's URL, Claude wrote 4 Wikipedia links from memory, which the
-        audit reported as `"text"` violations.
-      - `allow(["python.org"])` on OpenAI returned 27 sources, all on the list.
-
-**Verify while writing:** each example runs.
+Items 7–12 (truncated-call retry cost, empty truncated turns, made-up tool names in telemetry,
+tool-handler timeouts, `OTelObserver` redaction, and the stale 0.1-era docs) landed on
+`failure-telemetry`. The numbers stay so references remain valid.
+
+Left over:
+
+- **Empty `END_TURN` (item 8).** An empty non-truncated turn is still `ResponseInvalid` and is
+  retried verbatim. Decide whether it should end the conversation with empty text or be retried
+  once with a nudge.
+- **Hung tool threads (item 10).** A timed-out handler's thread is abandoned, not killed. A tool
+  that holds a lock or connection keeps it. `ToolSpec.timeout_seconds` defaults to `None`.
+- **Shared redaction helpers (item 11).** `events.py` and `tracing.py` each implement
+  redact-then-clip; move them to one module so they cannot drift.
+- **Hosted-tools guide examples (item 12).** The `agentloop-tools` parts of
+  `docs/guides/hosted-tools.md` (`native_web_search`, `DomainPolicy`) were written from this
+  backlog and not run here.
 
 ---
 

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -36,8 +36,27 @@ while `0.x`, a minor bump may break the public API (`agentloop.__all__`).
   `agentloop.provider.errors` counter (`error_type`, `phase`, `request_sent` of `true`,
   `false` or `unknown`). Tracing: `agentloop.attempts`, `gen_ai.response.id`, and
   `agentloop.error.*` on the model and root spans.
+- `ToolSpec.timeout_seconds` (and `@tool(timeout_seconds=...)`): a call that runs longer gets
+  an error result and the loop moves on. The handler's thread is abandoned, not killed.
+  Default `None` (no limit).
+- `OTelObserver(redact=..., allow_unredacted=...)`, with the event trace's rules.
+- `ConversationStatus.TOKEN_LIMIT` and `LoopBudgetExceeded(kind="max_tokens")`.
 
 ### Changed
+- `OTelObserver(capture_content=True)` now needs `redact` or `allow_unredacted=True`, else it
+  raises `ConfigurationError`. Captured text is redacted, then clipped; a redactor that raises
+  drops the attribute. Exceptions are recorded as an `exception` event with a redacted message
+  and no stack trace.
+- A turn truncated mid-tool-call no longer lowers a `max_tokens` above the 64,000 ceiling, and
+  when the budget cannot grow the run stops with `LoopBudgetExceeded(kind="max_tokens")`
+  (status `token_limit`) instead of re-billing the same cut-off up to `max_iterations`.
+- `ModelResponse.content` may be empty when `stop_reason` is `MAX_TOKENS`. The loop treats such
+  a turn (a reasoning model that spent the budget thinking) as a truncation and raises the
+  budget, instead of the adapter raising a retryable `ResponseInvalid`. Other empty turns are
+  still `ResponseInvalid`.
+- Calls to tools that are not registered are reported to observers, summaries, metrics, spans
+  and the event trace under `"<unregistered>"`, not the model's invented name. The model still
+  receives the real name in its "Unknown tool" result.
 - TLS verification, DNS and proxy failures are no longer retried: they fail the same way every
   time. They raise `ProviderUnreachable`, still a `ProviderUnavailable`, with `retryable = False`.
 - A 3xx is reported as `UnexpectedRedirect` (with the target host only), not as a retryable

diff --git a/docs/api/agent.md b/docs/api/agent.md
@@ -164,8 +164,9 @@ kept as internal state.
 
 `client` is borrowed: an agent constructed directly never closes it. `tools` accepts local
 `ToolSpec` objects and provider-run `HostedTool` objects. Values produced by `@tool` can be
-passed directly. Duplicate local tool names and unsupported hosted tools fail during
-construction. `deps` is the one runtime dependency bundle passed to local tool handlers,
+passed directly. Duplicate local tool names, unsupported hosted tools, and a local `ToolSpec` named like a
+hosted tool kind (for example `web_search`) fail during construction; native and client-side
+tools are separate, explicit choices. `deps` is the one runtime dependency bundle passed to local tool handlers,
 defaulting to `Deps()`. `trace=True` combines an `OTelObserver` with the supplied observer.
 `metrics=True` adds a `MetricsObserver`. `summary_options` configures the optional timeline,
 repeated-call detection, and limit warnings described in

diff --git a/docs/api/errors.md b/docs/api/errors.md
@@ -65,6 +65,10 @@ and `retry_wait_seconds`. Only `RateLimited`, `ProviderUnavailable`, `ProviderTi
 retryable: they are faults in the environment, and a retry fails the same way. A refused or
 reset connection and a read error stay retryable.
 
+`InsufficientCredits` covers OpenRouter's 402 and an OpenAI 429 whose error `type`/`code` is
+`insufficient_quota` or `credit_balance_exhausted`. The latter is a 429 only in name: it is
+not retried, since waiting does not add credit.
+
 An `AgentLoopError` raised from an active conversation also carries its
 `ConversationSummary` as `.summary`. Errors raised before a conversation starts may have
 `.summary = None`.

diff --git a/docs/api/loop.md b/docs/api/loop.md
@@ -34,8 +34,13 @@ run_conversation(
 
 Mutates and returns `transcript`. Each iteration sends a provider-independent
 `ModelRequest`, appends the response, executes requested tools, and appends their results. A
-response truncated mid-tool-call is discarded and retried with a doubled token allowance up
-to the ceiling.
+response truncated mid-tool-call, or truncated before producing anything readable, is discarded
+and retried with a doubled token allowance up to the 64,000 ceiling. A budget is never lowered,
+and when it cannot grow the run raises `LoopBudgetExceeded` with `kind="max_tokens"`.
+
+A provider that pauses a turn (`StopReason.PAUSED`, for example Anthropic's `pause_turn` while
+a hosted tool runs) is continued without sending a user message: the continuation joins the
+same assistant message, and each continuation counts as an iteration.
 
 The deadline is checked between turns. A provider call, its retries, or a started tool batch
 may finish after it. `None` disables the deadline. Exhaustion raises `LoopBudgetExceeded`;

diff --git a/docs/api/provider.md b/docs/api/provider.md
@@ -83,6 +83,16 @@ complete(request: ModelRequest) -> ModelResponse
 
 See [custom providers](../guides/custom-providers.md) for an implementation.
 
+## `supported_hosted_tools`
+
+```text
+supported_hosted_tools(client) -> frozenset[str]
+```
+
+The hosted tool kinds a client can run: its optional `hosted_tools` attribute, or an empty set
+when it has none. A client that says nothing supports nothing, so a custom provider needs no
+new attribute to stay an `LLMClient`.
+
 ## `HostedTool`
 
 A provider-run capability declared on each request. `kind` names the capability, such as