Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions examples/llmagent_with_streaming_progress_tool/.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Set TRPC_AGENT_API_KEY、TRPC_AGENT_BASE_URL、TRPC_AGENT_MODEL_NAME
TRPC_AGENT_API_KEY=your-api-key
TRPC_AGENT_BASE_URL=your-base-url
TRPC_AGENT_MODEL_NAME=your-model-name

67 changes: 67 additions & 0 deletions examples/llmagent_with_streaming_progress_tool/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Streaming Progress Tool

This example shows how to expose a **long-running tool that streams progress
events to the user in real time**, using `StreamingProgressTool`.

The wrapped function is an `async def` generator (`async def fn(...): yield ...`).
Every `yield` is surfaced to the runner as a `partial=True` Event tagged with
`custom_metadata={"tool_progress": True, ...}`. The **last** yielded value is
*also* the final `function_response` returned to the LLM.

```text
yield progress_1 --> partial Event (live)
yield progress_2 --> partial Event (live)
yield progress_3 --> partial Event (live) AND final function_response
```

This is different from the other two streaming-ish tools shipped with the SDK:

| Class | What gets streamed |
| --------------------------- | --------------------------------------------------- |
| `StreamingFunctionTool` | The *arguments* the LLM is generating for the call. |
| `LongRunningFunctionTool` | Nothing intermediate; just marks the call as slow. |
| **`StreamingProgressTool`** | The tool's *own* execution progress. |

## Run

```bash
cd examples/llmagent_with_streaming_progress_tool
cp ../mcp_tools/.env .env # or write your own
# edit .env to set TRPC_AGENT_API_KEY / BASE_URL / MODEL_NAME
python run_agent.py
```

Expected output (abridged):

```
User: Please crawl https://example.com and fetch the first 5 pages.
[crawl_site] ⏳ {'status': 'started', 'url': 'https://example.com', 'max_pages': 5}
[crawl_site] ⏳ {'status': 'fetched', 'page': 1, 'total': 5, ...}
[crawl_site] ⏳ {'status': 'fetched', 'page': 2, 'total': 5, ...}
...
[tool-result] crawl_site → {'status': 'done', 'url': '...', 'pages_fetched': 5, ...}
Assistant: I crawled example.com and fetched 5 pages. ...
```

## How to consume progress events on the client side

Filter on `event.partial` + `custom_metadata.tool_progress` to detect a
progress chunk. The raw value the tool yielded is available in
`custom_metadata['payload']` (for `dict`/`BaseModel` yields) and as a JSON
string in `event.content.parts[0].text` for plain-text consumers.

```python
async for event in runner.run_async(...):
meta = event.custom_metadata or {}
if event.partial and meta.get("tool_progress"):
print(meta["tool_name"], meta.get("payload") or event.get_text())
continue
# ...handle final events as usual
```

Notes:
- Progress events are NOT persisted into session history (they are partial).
- The LLM only ever sees the **last** yielded value as the tool response.
- If a batch contains a progress-streaming tool, the framework forces
sequential tool execution to keep interim events in deterministic order,
even if the agent has `parallel_tool_calls=True`.
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
#
# Copyright (C) 2026 Tencent. All rights reserved.
#
# tRPC-Agent-Python is licensed under Apache-2.0.
51 changes: 51 additions & 0 deletions examples/llmagent_with_streaming_progress_tool/agent/agent.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
#
# Copyright (C) 2026 Tencent. All rights reserved.
#
# tRPC-Agent-Python is licensed under Apache-2.0.
"""Agent that uses StreamingProgressTool for a long-running task."""

from trpc_agent_sdk.agents import LlmAgent
from trpc_agent_sdk.models import LLMModel
from trpc_agent_sdk.models import OpenAIModel
from trpc_agent_sdk.tools import StreamingProgressTool
from trpc_agent_sdk.types import GenerateContentConfig

from .config import get_model_config
from .prompts import INSTRUCTION
from .tools import crawl_site


def _create_model() -> LLMModel:
api_key, url, model_name = get_model_config()
return OpenAIModel(model_name=model_name, api_key=api_key, base_url=url)


def create_agent() -> LlmAgent:
"""Build the agent. ``crawl_site`` is wrapped in
``StreamingProgressTool(skip_summarization=True)`` so that:

1. Every ``yield`` becomes a partial Event the caller renders live.
2. The last ``yield`` is also the final ``function_response`` event –
persisted to the session as the canonical record of this turn.
3. ``skip_summarization=True`` makes :class:`LlmAgent` exit the
conversation loop immediately after the tool returns, so the LLM
is **not** asked to re-summarise the streamed output (which the
user has already seen).
"""
crawl_tool = StreamingProgressTool(crawl_site, skip_summarization=True)

return LlmAgent(
name="streaming_crawler",
description="Crawls a site step-by-step and streams progress to the user.",
model=_create_model(),
instruction=INSTRUCTION,
tools=[crawl_tool],
generate_content_config=GenerateContentConfig(
temperature=0.3,
max_output_tokens=1000,
),
)


root_agent = create_agent()
19 changes: 19 additions & 0 deletions examples/llmagent_with_streaming_progress_tool/agent/config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
#
# Copyright (C) 2026 Tencent. All rights reserved.
#
# tRPC-Agent-Python is licensed under Apache-2.0.
"""Agent config module."""

import os


def get_model_config() -> tuple[str, str, str]:
"""Get model config from environment variables."""
api_key = os.getenv('TRPC_AGENT_API_KEY', '')
url = os.getenv('TRPC_AGENT_BASE_URL', '')
model_name = os.getenv('TRPC_AGENT_MODEL_NAME', '')
if not api_key or not url or not model_name:
raise ValueError("TRPC_AGENT_API_KEY, TRPC_AGENT_BASE_URL, and TRPC_AGENT_MODEL_NAME "
"must be set in environment variables (e.g. via a .env file).")
return api_key, url, model_name
12 changes: 12 additions & 0 deletions examples/llmagent_with_streaming_progress_tool/agent/prompts.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
#
# Copyright (C) 2026 Tencent. All rights reserved.
#
# tRPC-Agent-Python is licensed under Apache-2.0.
"""Prompts for the streaming-progress tool demo."""

INSTRUCTION = (
"You are a helpful crawling assistant. When the user asks you to crawl, fetch, "
"or inspect a website, ALWAYS call the `crawl_site` tool. Pass a sensible "
"`max_pages` (default to 5 if unspecified). After the tool finishes, "
"summarise what was fetched in 1-2 sentences in the user's language.")
55 changes: 55 additions & 0 deletions examples/llmagent_with_streaming_progress_tool/agent/tools.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
#
# Copyright (C) 2026 Tencent. All rights reserved.
#
# tRPC-Agent-Python is licensed under Apache-2.0.
"""A long-running tool that streams progress events to the user.

The tool simulates a multi-step site crawl. Each step yields a structured
progress payload that the framework surfaces as a partial Event in real time.
The **last** yielded value is also the final tool response fed back to the LLM.
"""

from __future__ import annotations

import asyncio
import random
from typing import AsyncIterator


async def crawl_site(url: str, max_pages: int = 5) -> AsyncIterator[dict]:
"""Crawl ``url`` and stream progress for every page fetched.

Use this for long-running fetches where the user benefits from seeing
incremental progress instead of staring at a spinner.

Args:
url: The site URL to crawl (any string for demo purposes).
max_pages: How many pages to simulate fetching. Defaults to 5.

Yields:
dict: One progress payload per step. The final payload is also the
return value the LLM sees.
"""
yield {"status": "started", "url": url, "max_pages": max_pages}

fetched_titles: list[str] = []
for page_index in range(1, max_pages + 1):
# Simulate variable per-page latency so the streaming is observable.
await asyncio.sleep(random.uniform(0.4, 1.2))
title = f"{url} - page {page_index}"
fetched_titles.append(title)
yield {
"status": "fetched",
"page": page_index,
"total": max_pages,
"title": title,
"progress": round(page_index / max_pages, 2),
}

yield {
"status": "done",
"url": url,
"pages_fetched": len(fetched_titles),
"titles": fetched_titles,
}
105 changes: 105 additions & 0 deletions examples/llmagent_with_streaming_progress_tool/run_agent.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
#
# Copyright (C) 2026 Tencent. All rights reserved.
#
# tRPC-Agent-Python is licensed under Apache-2.0.
"""Demo: streaming progress events from a long-running tool.

Run with::

cd examples/llmagent_with_streaming_progress_tool
python run_agent.py

Make sure ``TRPC_AGENT_API_KEY``, ``TRPC_AGENT_BASE_URL`` and
``TRPC_AGENT_MODEL_NAME`` are set in your environment or .env file.
"""

import asyncio
import sys
import uuid
from pathlib import Path

from dotenv import load_dotenv
from trpc_agent_sdk.runners import Runner
from trpc_agent_sdk.sessions import InMemorySessionService
from trpc_agent_sdk.types import Content
from trpc_agent_sdk.types import Part

load_dotenv()

sys.path.append(str(Path(__file__).parent))


async def run_streaming_progress_demo() -> None:
"""Issue one query and pretty-print every event surfaced by the runner.

Three event kinds matter here:
1. ``event.partial`` with ``custom_metadata.tool_progress`` → a *progress*
chunk from the streaming tool. Print it live; do NOT treat it as a
tool response.
2. ``event.partial`` text from the LLM (no ``tool_progress`` marker) →
streaming model output.
3. ``partial=False`` events with a ``function_response`` part → the final
tool result; with a ``text`` part → the model's final reply.
"""

app_name = "streaming_progress_demo"
from agent.agent import root_agent

session_service = InMemorySessionService()
runner = Runner(app_name=app_name, agent=root_agent, session_service=session_service)

user_id = "demo_user"
session_id = str(uuid.uuid4())
await session_service.create_session(app_name=app_name, user_id=user_id, session_id=session_id)

query = "Please crawl https://example.com and fetch the first 5 pages."
print("=" * 60)
print(f"User: {query}")
print("=" * 60)

user_content = Content(parts=[Part.from_text(text=query)])

async for event in runner.run_async(user_id=user_id, session_id=session_id, new_message=user_content):
meta = event.custom_metadata or {}

# --- 1. Tool progress (partial, comes from StreamingProgressTool) ---
if event.partial and meta.get("tool_progress"):
payload = meta.get("payload")
tool_name = meta.get("tool_name", "?")
print(f"[{tool_name}] ⏳ {payload if payload is not None else event.get_text()}")
continue

if not event.content or not event.content.parts:
continue

# --- 2. Streaming LLM text ---
if event.partial:
for part in event.content.parts:
if part.text:
print(part.text, end="", flush=True)
continue

# --- 3. Final events ---
for part in event.content.parts:
if part.function_call:
print(f"\n[tool-call] {part.function_call.name}({part.function_call.args})")
elif part.function_response:
print(f"\n[tool-result] {part.function_response.name} → "
f"{part.function_response.response}")
elif part.text:
print(f"\nAssistant: {part.text}")

print("\n" + "-" * 60)


if __name__ == "__main__":
print("""
+--------------------------------------------------------------+
| StreamingProgressTool Demo (long-running tool) |
| |
| Watch the tool yield progress events live, then the LLM |
| summarises the final result. |
+--------------------------------------------------------------+
""")
asyncio.run(run_streaming_progress_demo())
Loading
Loading