From c309c9b019e38284d1b123b8fc5c6cd380bb1351 Mon Sep 17 00:00:00 2001 From: aaron Date: Tue, 20 Oct 2026 22:49:45 -0700 Subject: [PATCH 1/2] Stop tracking internal notes and personal tooling .claude/ and DESIGN.md stay on disk but are ignored, ahead of open-sourcing the repo. Co-Authored-By: Claude Sonnet 5.5 --- .claude/skills/release/SKILL.md | 71 ------------- .gitignore | 4 + DESIGN.md | 183 -------------------------------- 3 files changed, 4 insertions(+), 254 deletions(-) delete mode 100644 .claude/skills/release/SKILL.md delete mode 100644 DESIGN.md diff --git a/.claude/skills/release/SKILL.md b/.claude/skills/release/SKILL.md deleted file mode 100644 index 1658579..0000000 --- a/.claude/skills/release/SKILL.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -name: release -description: Cut a PyPI release of based-models-agentloop-tools by pushing a version tag. Use only when the user invokes /release. -disable-model-invocation: true ---- - -# Release based-models-agentloop-tools to PyPI - -Distribution `based-models-agentloop-tools`, import name `agentloop_tools`, repo `based-models/agent-loop-tools`. - -Pushing a `vX.Y.Z` tag **publishes to PyPI immediately** (Trusted Publishing, no approval -gate), and a PyPI version number can **never be reused**, even after deletion. Everything -before the tag is preparation; the tag is the point of no return. The full process notes live in the `agent-loop` repo's `docs/guides/releasing.md`; this repo -uses the same workflow. - -## 1. Preconditions (stop and report on any failure; do not fix silently) -Run read-only checks first: -- On `main`, up to date with `origin/main` (`git fetch`, then compare). Unrelated local - edits that do not belong in the release: leave them alone and say so. -- CI on the release commit is green: `gh run list --branch main --limit 3`. -- Version `X.Y.Z` is identical in `pyproject.toml` and `src/agentloop_tools/__init__.py` - (`__version__`). The release workflow rejects a mismatch, but catch it earlier. -- `CHANGELOG.md` has a `## [X.Y.Z]` heading for exactly this version. -- No `vX.Y.Z` tag exists locally or on origin (`git ls-remote --tags origin`), and - `https://pypi.org/pypi/based-models-agentloop-tools/X.Y.Z/json` is a 404 (that version is unpublished). -- The `based-models-agentloop` range in `pyproject.toml` is satisfied by a version already on - PyPI (`https://pypi.org/pypi/based-models-agentloop/json`). Its CI installs that dependency - from PyPI, so an unpublished requirement means red CI. Release `agent-loop` first if needed. -- README or doc changes meant for the PyPI page are **merged to `main`**, because the tag - publishes what is on `main`. - -If the version is not bumped yet, offer to do the bump (pyproject, `__init__`, changelog) on a -branch and open a PR. Never push version bumps straight to `main`. - -## 2. Confirm before tagging -Tell the user exactly what will happen and wait for a clear yes in this conversation: -> Tagging `vX.Y.Z` on `agent-loop-tools` at commit `` will publish `based-models-agentloop-tools X.Y.Z` to PyPI -> immediately. That version can never be re-uploaded. Proceed? - -A yes to an earlier question, or to a different release, does not count. - -## 3. Tag (only after the yes) -```bash -git tag vX.Y.Z && git push origin vX.Y.Z # one tag, by name; never `git push --tags` -``` - -## 4. Watch, then verify -- Find the run: `gh run list --workflow release.yml --limit 1`, then wait until it - completes (a background watch, not a sleep loop). Report each job: `build`, - `publish-pypi`, `github-release`. -- Verify from the real index in a throwaway venv outside the repo: - ```bash - python3 -m venv "$TMP/v" && "$TMP/v/bin/pip" install based-models-agentloop-tools "based-models-agentloop[anthropic]" - "$TMP/v/bin/python" -c "import agentloop_tools; print(agentloop_tools.__version__)" - ``` -- Confirm the GitHub Release has the wheel, sdist and `SHA256SUMS`: - `gh release view vX.Y.Z --json assets`. - -## 5. If something fails -- Fails before `publish-pypi` uploads (tests, version check, missing pending publisher): - nothing was published and the version is not used. Fix, then delete and re-push the tag - **only if the user agrees**, since re-pushing re-triggers a publish. -- `publish-pypi` succeeded but the release is bad: **do not** move or delete the tag. - Recommend yanking `X.Y.Z` on PyPI (the user does this in the PyPI web UI) and releasing - `X.Y.(Z+1)`. - -## Never -- Tag without the explicit confirmation in step 2. -- `git push --tags`, force-push a tag, or reuse a version number. -- Add API tokens or `twine upload`: publishing is only via the workflow. -- Release from anything but an up-to-date `main`. diff --git a/.gitignore b/.gitignore index b732b5b..5692f7a 100644 --- a/.gitignore +++ b/.gitignore @@ -7,3 +7,7 @@ dist/ # Local-only: experiment scripts, their output, and the reference captures. experiments/ + +# Internal notes and personal tooling: kept locally, not published +.claude/ +DESIGN.md diff --git a/DESIGN.md b/DESIGN.md deleted file mode 100644 index 5a85dba..0000000 --- a/DESIGN.md +++ /dev/null @@ -1,183 +0,0 @@ -# agentloop-tools: design record - -The decisions behind this library and the agentloop 0.2 seam it depends on, with the -reason for each. - -## Layers - -``` -harness app: prompt, CLI, secrets, approval policy, WHICH tools are on -agentloop-tools mechanism: tool factories, validation, readers -agentloop the loop, the provider seam, transcript; no tools -``` - -Each layer depends only on the one below. This library imports agentloop's public API -(`agentloop.__all__`) and never reads configuration; `tests/test_layering.py` enforces -both rules. - -## Native search is a HOSTED tool, so part of it lives in agentloop - -The provider runs a native search, so the request must declare it and the transcript -must carry what came back. Only agentloop's adapters can do either. agentloop 0.2 -therefore learned one generic idea, *hosted tools*, and nothing web-specific: - -| agentloop (generic) | agentloop-tools (web-specific) | -|---|---| -| `HostedTool(kind, options)` on `ModelRequest.hosted_tools` | `native_web_search(...)`: typed, validated options | -| Each adapter translates `kind="web_search"` | Domain normalization; ASCII-only (homograph) check | -| `HostedToolPart`, `Source`, `Citation` (a readable view) | `sources(transcript)`, `citations(transcript)` | -| `StopReason.PAUSED` + loop continuation | | -| `Agent` refuses a hosted kind the client cannot run | | - -Deliberately *not* in agentloop: no observer event (observers scan `on_response`), no -usage field (count the parts), no typed web options. - -**Native and client search are separate first-class choices.** Neither is a fallback: -`Agent` raises, naming the provider, if the provider cannot run a hosted tool. Client -search is on agentloop's BACKLOG. - -| | Native (`native_web_search`) | Client (backlog) | -|---|---|---| -| Who searches | The provider | Your code, through a search API | -| Providers | Anthropic, OpenAI Responses | All of them | -| What you see | Queries, URLs, citations; page text is encrypted | Full snippets, as a tool result | -| Approval | Per request: declare it or don't, plus domain filters | Per call, through `before_tool` | - -## Replay: `Message.origin` - -Both providers need the assistant turn sent back **exactly**: -- **Anthropic:** content blocks go back as received. Altering or omitting - `encrypted_content` or a citation's `encrypted_index` returns a 400. Dynamic filtering - adds code-execution blocks. A mixed client/server turn returns the search result in the - *next* response. -- **OpenAI Responses:** `reasoning` items (with `encrypted_content`) and each message's - `phase` must survive, in order. - -Canonical parts cannot rebuild any of that. So each adapter stores its own wire form on -the message: `Origin(provider, api, model, raw)`. -- **Same `(provider, api)`:** the adapter replays `raw`. -- **Anything else:** it rebuilds from the parts and drops what cannot move (reasoning, - hosted activity). - -The parts are the portable view; `raw` is same-provider fidelity. Unknown block types -survive automatically. They used to be dropped silently. - -The same mechanism implements OpenRouter reasoning replay. The Completions codec -reattaches `reasoning_details` from the origin, but rebuilds the message itself from -parts, because generated tool-call ids must match the results. - -## OpenAI: Completions and Responses, one transport - -`OpenAICompatClient` is the transport: auth, timeouts, HTTP errors, the non-JSON guard, -`close`. A `Codec` is the wire format: `path`, `request_to_wire`, `response_from_wire`, -`hosted_tools`. -- **`api="completions"`:** `/chat/completions`. OpenRouter, Modal, and OpenAI when asked. - It refuses hosted tools rather than dropping them. -- **`api="responses"`:** `/responses`, OpenAI's default (`OPENAI_API`). This is where - hosted search lives. - -There is no duplicated transport and no mode flags inside the translators. - -## Responses is stateless, always - -Responses can hold the conversation server-side: `store` defaults to **true**, and there -are `previous_response_id` and Conversations. agentloop's `Transcript` is the single -source of truth: JSONL resume, discarded truncated turns, provider switching, and blind -retries of a pure `complete()`. A second history would drift from it. - -So every request: -- sends `store: false` and never sends `previous_response_id`; -- resends the full transcript as `input`, as Completions and Anthropic already do; -- strips item `id`s on replay. With nothing persisted, an id makes the server look up an - item that doesn't exist, and the request fails. - -`web_search_call` items are not replayed; the message's `url_citation` annotations carry -the evidence. - -## Option policy (every adapter) - -- **Unknown option:** error. -- **Unsupported restriction** (a domain list): error, because dropping it would widen - access. -- **Unsupported tuning** (`max_uses` on OpenAI, `search_context_size` on Anthropic): - logged and ignored. - -## Domain policy: prevent and report, nothing more - -`DomainPolicy.block([...])` or `DomainPolicy.allow([...])` is one list with two jobs: - -- **Prevent:** `policy.native_web_search()` sends the list to the provider as its search - filter, on both providers. -- **Report:** `policy.audit(transcript, since=n)` lists every `Violation` where a finished - turn still reached a domain the policy does not permit. It checks sources (a provider - leak), citations, and the answer text. - -It exists because a provider filter is not a guarantee. Mirrors, subdomain edge cases, -and the model's own knowledge all get past it. Live, with Wikipedia blocked, Claude still -wrote four Wikipedia URLs from memory when asked; the audit caught all four. - -**The hard limit:** with native search the model reads the results inside the provider's -response. Nothing downstream can stop it reading a page; the policy can only prevent what -the provider filters and report what came out. The real guarantee belongs to client-side -search, which will call `policy.permits(url)` on each hit before the model sees it -(BACKLOG). - -**Deliberately minimal:** it doesn't hide, redact, retry, raise, or touch the transcript. -Altering replayed search results would 400 on Anthropic anyway. What to do about a -violation is the developer's decision. - -**Text-scan boundaries:** -- **Full URLs, in both modes:** markdown targets are included, and balanced parentheses - are kept, as in Wikipedia's `_(programming_language)`. -- **Bare host mentions, in block mode only:** word-bounded per rule, so - `notwikipedia.org` doesn't match. -- **Why allow mode skips bare hosts:** it would need a generic host pattern, which would - also flag `config.py`. - -## Live verification (September 2026) - -Confirmed live against Anthropic (`claude-opus-5`), OpenAI Responses (`gpt-5.5`) and -OpenRouter (`openrouter/free`): - -- **Anthropic:** search (run with dynamic filtering, then the default; see below). The follow-up turn replayed the encrypted - results from `origin.raw` and was accepted. A successful code-execution result is an - object, not an error; the parser was corrected to count only `*_error` / `error_code`. -- **OpenAI Responses:** search with sources and `url_citation`s. Follow-up turns - replayed `message` items (with `phase`), omitting `web_search_call` items, with every - id stripped under `store: false`, and were accepted. A `reasoning` item was replayed - id-less in two later requests and accepted. Three parallel function calls and their - outputs round-tripped. -- **Domain filters:** `blocked_domains=["wikipedia.org"]` was accepted AND honored by both - providers: no Wikipedia source on a query it would normally dominate. -- **Option policy:** `max_uses` on OpenAI is logged and ignored, as designed. -- **OpenRouter:** a follow-up routed to a different upstream model accepted the replayed - `reasoning_details`. -- **OpenAI errors:** an empty balance is a 429 with `type: insufficient_quota`, - `code: credit_balance_exhausted`. It is now `InsufficientCredits` and not retried. - -Not triggered live: an Anthropic `pause_turn` (a merged continuation is covered by fixtures). - -## Anthropic `dynamic_filtering` is off by default, because it removes citations - -`native_web_search(dynamic_filtering=False)` is the default, so the adapter sends basic -search (`web_search_20250305`). This was flipped after an experiment -(`claude-opus-5`, 2 questions × 3 variants, 6 requests): - -| Variant | Citations | How the model saw results | -|---|---|---| -| `web_search_20260209`, dynamic filtering | 0, 0 | Printed output of a program it wrote | -| `web_search_20250305`, basic | 2, 3 | Search-result blocks | -| `web_search_20260209` + `allowed_callers: ["direct"]` | 2, 5 | Search-result blocks | - -With filtering on, Claude runs code that calls the search and prints a filtered excerpt, -and it reads only that (encrypted) stdout. A citation must point at a search-result block -in context, so there is nothing to cite. The text arrives split where citations would -sit, each block with `citations: []`, and the model lists sources in prose instead. - -- **It's the code path, not the tool version:** the direct-caller variant uses the same - version and cites normally. -- **It didn't save tokens here:** single, simple searches cost slightly more input with - filtering (13,300 vs 12,827). -- **What still works:** `sources()` is filled in every mode. -- **The trade-off:** filtering is worth turning on only for search-heavy work where - citations aren't needed. From fb023a04b6d91c05d0f86c8ea600b45a261f9d63 Mon Sep 17 00:00:00 2001 From: aaron Date: Tue, 20 Oct 2026 22:58:33 -0700 Subject: [PATCH 2/2] Fill .gitignore gaps and drop references to untracked notes Ignores .env.* (keeping .env.example); agentloop-tools gains the standard entries agent-loop already had. Two source comments no longer point at files that are not in the repo. Co-Authored-By: Claude Sonnet 5.5 --- .gitignore | 22 ++++++++++++++++++++++ src/agentloop_tools/web.py | 2 +- 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/.gitignore b/.gitignore index 5692f7a..68390e6 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,6 @@ +.env +.env.* +!.env.example __pycache__/ *.egg-info/ dist/ @@ -11,3 +14,22 @@ experiments/ # Internal notes and personal tooling: kept locally, not published .claude/ DESIGN.md + +.coverage +.coverage.* +htmlcov/ + +# notebooks and their checkpoints +.ipynb_checkpoints/ + +# build output and virtualenvs +build/ +.venv/ +venv/ + +# editor and OS files +.DS_Store +.idea/ + +# personal editor settings +.vscode/ diff --git a/src/agentloop_tools/web.py b/src/agentloop_tools/web.py index 907a0a1..2e9bea4 100644 --- a/src/agentloop_tools/web.py +++ b/src/agentloop_tools/web.py @@ -6,7 +6,7 @@ refused at `Agent` construction by a provider that cannot run it. Client-side search -- your own code calling a search API -- is a separate, -first-class choice, not a fallback for this one. It is on agentloop's BACKLOG. +first-class choice, not a fallback for this one. It is planned for a later release. """ from __future__ import annotations