Add agentloop-tools documentation and refresh README

README.md

@@ -39,8 +39,9 @@ pip install based-models-agentloop-tools "based-models-agentloop[openai-compat]"
 ```
 
 Native web search works with Anthropic Messages and the OpenAI Responses API. OpenAI uses
-Responses by default; it will not work when `OPENAI_API=completions` is selected. OpenRouter,
-Modal/vLLM, and other Chat Completions-compatible providers do not offer this hosted tool.
+Responses by default once `LLM_PROVIDER=openai` is selected; it will not work when
+`OPENAI_API=completions` is selected. OpenRouter, Modal/vLLM, and other Chat
+Completions-compatible providers do not offer this hosted tool.
 
 ## Quick start
 
@@ -54,19 +55,18 @@ from agentloop import Agent, Transcript, final_text
 from agentloop_tools.web import citations, native_web_search, sources
 
 search = native_web_search(
-    allowed_domains=["europa.eu"],
-    user_location={"country": "BE"},
+    allowed_domains=["python.org"],
 )
 
 transcript = Transcript()
 transcript.add_user_message(
-    "What does the EU AI Act say about general-purpose models?"
+    "Search the Python documentation and explain how to create a virtual environment."
 )
 
 with Agent.from_env(
-    os.environ,
+    {**os.environ, "LLM_PROVIDER": "openai"},
     tools=[search],
-    system="Use web search and cite your sources.",
+    system="Use web search before answering. Cite the pages you use.",
 ) as agent:
     agent.converse(transcript)
 
@@ -106,7 +106,7 @@ search = native_web_search(
 | `blocked_domains` | Exclude these domains and paths | Yes | Yes |
 | `user_location` | Approximate city, region, country, or timezone | Yes | Yes |
 | `max_uses` | Maximum searches per request | Yes | Ignored with a warning |
-| `dynamic_filtering` | Filter results with provider-side code | Yes | Ignored with a warning |
+| `dynamic_filtering` | Filter results with provider-side code (Claude 4.6+) | Yes | Ignored with a warning |
 | `search_context_size` | Amount of search content supplied to the model | Ignored with a warning | Yes |
 
 Unknown options are rejected. Unsupported domain restrictions are never silently dropped;
@@ -120,7 +120,8 @@ Domain filters use bare ASCII domains without a URL scheme:
 native_web_search(allowed_domains=["europa.eu", "example.com/reports"])
 ```
 
-- Pass either `allowed_domains` or `blocked_domains`, never both.
+- Pass either `allowed_domains` or `blocked_domains`, never both. Passing both raises
+  `ValueError`.
 - A rule includes the named host and its subdomains.
 - An optional path restricts the rule to that path prefix.
 - `*` is allowed within a path, but not in a hostname.
@@ -151,8 +152,9 @@ it, and not every search mode produces citations.
 ### Anthropic dynamic filtering
 
 `dynamic_filtering=False` is the default because Anthropic's basic search preserves
-citations. When enabled, Anthropic runs provider-side code to filter search results before
-they reach the model. This can help search-heavy requests, but it has important tradeoffs:
+citations. With Claude 4.6 or later, enabling it runs provider-side code to filter
+results before they reach the model. This can help search-heavy requests, but it has
+important tradeoffs:
 
 - `sources()` still returns the pages consulted.
 - `citations()` may be empty because the model reads the filtering program's output rather
@@ -172,14 +174,20 @@ import os
 from agentloop import Agent, Transcript, final_text
 from agentloop_tools.web import DomainPolicy
 
-policy = DomainPolicy.allow(["europa.eu", "oecd.org"])
-search = policy.native_web_search(search_context_size="high")
+policy = DomainPolicy.allow(["python.org"])
+search = policy.native_web_search()
 
 transcript = Transcript()
 before = len(transcript.messages)
-transcript.add_user_message("Summarize recent AI-policy guidance.")
+transcript.add_user_message(
+    "Search the Python documentation and explain how to create a virtual environment."
+)
 
-with Agent.from_env(os.environ, tools=[search]) as agent:
+with Agent.from_env(
+    {**os.environ, "LLM_PROVIDER": "openai"},
+    tools=[search],
+    system="Use web search before answering. Cite the pages you use.",
+) as agent:
     agent.converse(transcript)
 
 violations = policy.audit(transcript, since=before)
@@ -193,7 +201,7 @@ else:
 Construct policies with one of:
 
 ```python
-allowed = DomainPolicy.allow(["europa.eu", "oecd.org"])
+allowed = DomainPolicy.allow(["python.org"])
 blocked = DomainPolicy.block(["example.com", "example.org/private"])
 ```
 
@@ -250,13 +258,16 @@ Tool families are intentionally imported explicitly rather than re-exported from
 root. The package uses only `agentloop`'s public API and never reads configuration, API keys,
 or environment variables.
 
-This release provides provider-hosted web search. Client-side search—where your own Python
-tool calls a search API and controls results before the model sees them—is a separate feature
+This release provides provider-hosted web search. Client-side search, where your own Python
+tool calls a search API and controls results before the model sees them, is a separate feature
 and is not included yet.
 
 ## Documentation
 
-Full documentation and additional examples are coming soon.
+Read the [documentation](https://github.com/based-models/agent-loop-tools/tree/main/docs) for
+the [hosted search guide](https://github.com/based-models/agent-loop-tools/blob/main/docs/native-web-search.md),
+[sources and citations](https://github.com/based-models/agent-loop-tools/blob/main/docs/evidence.md),
+and [domain policies](https://github.com/based-models/agent-loop-tools/blob/main/docs/domain-policy.md).
 
 ## Stability and license
 

docs/domain-policy.md

@@ -0,0 +1,71 @@
+# Domain policies
+
+`DomainPolicy` keeps one allowlist or blocklist for two steps. First,
+`policy.native_web_search()` sends its domains to the provider's search tool. After
+the response, `policy.audit(transcript)` reports any disallowed domains found in
+assistant search sources, citations, or answer text.
+
+## Create and use a policy
+
+```python
+from agentloop_tools.web import DomainPolicy
+
+policy = DomainPolicy.allow(["python.org"])
+search = policy.native_web_search()
+
+print(search.options)
+print(policy.permits("https://docs.python.org/3/"))
+print(policy.permits("https://example.org/"))
+```
+
+Output:
+
+```text
+{'allowed_domains': ['python.org']}
+True
+False
+```
+
+Use `search` in an `Agent`'s `tools` list, as in the
+[native search guide](native-web-search.md). For a blocklist, call
+`DomainPolicy.block(["example.org"])`; other domains remain permitted.
+`policy.match(url)` returns the rule that matched or `None`. A host rule includes
+subdomains, and an optional path limits the rule to that path prefix. Passing
+domain options to `policy.native_web_search()` raises `ValueError` because the
+policy already supplies them.
+
+## Audit an answer
+
+```python
+from agentloop import Transcript
+from agentloop.transcript import TextPart
+from agentloop_tools.web import DomainPolicy
+
+policy = DomainPolicy.allow(["python.org"])
+transcript = Transcript()
+transcript.add_user_message("Summarize the topic.")
+before = len(transcript.messages)
+transcript.add_assistant_message([
+    TextPart(text="See https://example.org/guide for details."),
+])
+
+for violation in policy.audit(transcript, since=before):
+    print(violation.where, violation.url, violation.rule)
+```
+
+Output:
+
+```text
+text https://example.org/guide None
+```
+
+`since` is the transcript message index to start scanning, useful when auditing
+only the latest turn. Each `Violation` has `where`, `url`, `message_index`, and
+`rule` fields. `rule` is the matching blocklist entry in block mode and `None`
+for a URL outside an allowlist. An allowlist audit checks full URLs in answer
+text; a blocklist audit also checks bare mentions of blocked hosts.
+
+The audit reads the transcript and returns violations. Your application decides
+how to handle them. Because the provider performs native search inside its own
+response, the audit cannot stop the model from reading a page returned by the
+provider. See [sources and citations](evidence.md) for the evidence it inspects.

docs/evidence.md

@@ -0,0 +1,46 @@
+# Sources and citations
+
+`sources(transcript)` returns the pages found in hosted search results, once per URL
+in first-appearance order. `citations(transcript)` returns citations attached to
+assistant text in transcript order. A searched page may never be cited.
+
+The same readers work on a transcript from a live agent run. This short example
+constructs the relevant transcript parts so their different results are easy to see:
+
+```python
+from agentloop import Citation, HostedToolPart, Source, Transcript
+from agentloop.transcript import TextPart
+from agentloop_tools.web import citations, sources
+
+transcript = Transcript()
+transcript.add_user_message("Summarize the source.")
+transcript.add_assistant_message([
+    HostedToolPart(
+        kind="web_search",
+        sources=(
+            Source(url="https://example.org/a"),
+            Source(url="https://example.org/b"),
+            Source(url="https://example.org/a"),
+        ),
+    ),
+    TextPart(
+        text="A short summary.",
+        citations=(Citation(url="https://example.org/b"),),
+    ),
+])
+
+print([source.url for source in sources(transcript)])
+print([citation.url for citation in citations(transcript)])
+```
+
+Output:
+
+```text
+['https://example.org/a', 'https://example.org/b']
+['https://example.org/b']
+```
+
+`sources()` reads hosted tool parts. `citations()` reads citation data on assistant
+content. Both return empty lists when that evidence is absent. Anthropic dynamic
+filtering may produce sources without citations because the model sees the filtering
+program's output instead of citation-bearing search results.

docs/index.md

@@ -0,0 +1,18 @@
+# agentloop-tools documentation
+
+`agentloop-tools` adds typed tool helpers to `agentloop`. Its current tool family
+configures provider-hosted web search and reads the evidence recorded in a transcript.
+The provider runs the search. `agentloop` sends the tool configuration and records the
+response; `agentloop-tools` validates options and helps inspect the result.
+
+Start with the [README](../README.md) for installation and a complete agent example.
+Then use these guides:
+
+- [Native web search](native-web-search.md) explains the tool factory, provider support,
+  and search options.
+- [Sources and citations](evidence.md) shows how to read search evidence from a
+  `Transcript`.
+- [Domain policies](domain-policy.md) explains allowlists, blocklists, and auditing.
+
+Import this tool family from `agentloop_tools.web`. The package root does not re-export
+its functions or classes.

docs/native-web-search.md

@@ -0,0 +1,84 @@
+# Native web search
+
+`native_web_search()` creates an `agentloop.HostedTool` for a provider to execute.
+Pass it in an `Agent`'s `tools` list. The provider searches, while `agentloop`
+records the search result in the conversation transcript.
+
+## Run an agent with search
+
+Install `based-models-agentloop-tools` and the agentloop extra for your provider, as
+shown in the [README](../README.md#installation). Set the provider's API key in your
+environment. This example uses OpenAI Responses; Anthropic Messages also supports
+native web search.
+
+```python
+import os
+
+from agentloop import Agent, Transcript, final_text
+from agentloop_tools.web import native_web_search
+
+search = native_web_search(allowed_domains=["python.org"])
+transcript = Transcript()
+transcript.add_user_message(
+    "Search the Python documentation and explain how to create a virtual environment."
+)
+
+with Agent.from_env(
+    {**os.environ, "LLM_PROVIDER": "openai"},
+    tools=[search],
+    system="Use web search before answering. Cite the pages you use. Answer in one sentence.",
+) as agent:
+    agent.converse(transcript)
+
+print(final_text(transcript))
+```
+
+Output:
+
+```text
+According to the Python docs, create a virtual environment by running `python -m venv .venv` in your project directory, then activate it with `source .venv/bin/activate` on macOS/Linux or `.venv\Scripts\activate.bat` / `.venv\Scripts\Activate.ps1` on Windows; the environment is isolated from the base Python by default and `pip` is bootstrapped unless you pass `--without-pip`. ([docs.python.org](https://docs.python.org/3/library/venv.html))
+```
+
+For OpenAI, use the Responses API, which is the default. Setting
+`OPENAI_API=completions` makes hosted search unavailable. OpenRouter, Modal/vLLM,
+and other Chat Completions providers do not support this hosted tool.
+
+## Configure search
+
+```python
+from agentloop_tools.web import native_web_search
+
+search = native_web_search(
+    allowed_domains=["python.org"],
+    search_context_size="medium",
+)
+print(search.kind)
+print(search.options)
+```
+
+Output:
+
+```text
+web_search
+{'allowed_domains': ['python.org'], 'search_context_size': 'medium'}
+```
+
+| Option | Meaning | Anthropic | OpenAI Responses |
+| --- | --- | --- | --- |
+| `allowed_domains` | Include only these hosts or paths | Supported | Supported |
+| `blocked_domains` | Exclude these hosts or paths | Supported | Supported |
+| `user_location` | Approximate city, region, country, or timezone | Supported | Supported |
+| `max_uses` | Limit searches per request | Supported | Ignored with a warning |
+| `dynamic_filtering` | Filter results with provider-side code, Claude 4.6+ | Supported | Ignored with a warning |
+| `search_context_size` | Set how much search content reaches the model | Ignored with a warning | Supported |
+
+Use either `allowed_domains` or `blocked_domains`. Passing both raises `ValueError`.
+Domain entries are bare ASCII hosts, optionally followed by a path such as
+`example.org/reports`; schemes such as `https://` are rejected. A host rule also
+covers its subdomains. `*` is allowed in paths, but not in hostnames.
+
+`dynamic_filtering` defaults to `False`. Anthropic's basic search can preserve
+citations. With Claude 4.6 or later, dynamic filtering runs provider-side code on
+results, which can leave the answer without citation blocks. It is not eligible for
+Zero Data Retention. See [sources and citations](evidence.md) for reading both forms
+of evidence.