Skip to content

Make the agent tool-call budget configurable (env var; no UI/config today) #47

Description

@gangtao

Enhancement

Make the agent's tool-call budget configurable so operators can control how much the chat agent explores per question, instead of it being hardcoded. Today a user hits "I've reached my exploration budget" and there is no env var and no management UI to raise or lower it — the only way to change it is editing source.

Tech investigation — there are two coupled limits, both hardcoded

  1. Soft budget (the message users see)src/tpk/agent.py:132, inside system_prompt():

    "You have a hard budget of about 12 tool calls per question — track your count, and once you're close to that budget, STOP exploring and write your best answer…"

    This is a literal string in the prompt. The model self-limits and reports "I've reached my exploration budget" — it's not enforced in code, just instructed. The number 12 is hardcoded in the prompt text and would need to be interpolated from config.

  2. Hard cap (enforced ceiling)src/tpk/agent.py:66 RECURSION_LIMIT = 40, passed as config={"recursion_limit": RECURSION_LIMIT} in src/tpk/server.py:236. LangGraph counts each step (~2 per tool call: model turn + tool turn), so 40 ≈ ~20 tool calls. Exceeding it raises GraphRecursionError, surfaced as an error event.

The two must stay consistent: the soft budget should sit safely below the hard cap (roughly recursion_limit ≈ 2 × tool_budget + headroom), or the model will keep exploring past the soft budget and get cut off hard.

Proposed change

Add env-var config, matching the existing TPK_AGENT_* / TPK_CHAT_AUDIT pattern (AgentConfig.from_env() in src/tpk/config.py:127):

  • TPK_AGENT_TOOL_BUDGET (default 12) — the soft per-question tool-call budget. system_prompt() interpolates this number instead of the literal "about 12 tool calls".
  • TPK_AGENT_RECURSION_LIMIT (default 40) — the LangGraph hard cap. Replaces the module constant RECURSION_LIMIT.

Carry both on AgentConfig (or read where the prompt/agent is built) so they flow through build_agent/system_prompt and the /chat astream_events config.

Guardrails:

  • Validate TPK_AGENT_TOOL_BUDGET ≥ 1 and TPK_AGENT_RECURSION_LIMIT ≥ some floor; on an out-of-range/unparseable value, fall back to the default (don't crash startup).
  • If only the tool budget is raised without the recursion limit, either auto-derive a safe recursion_limit (e.g. max(configured, 2*tool_budget + 4)) or log a warning — so raising the soft budget doesn't just move users from the graceful "best answer" path onto the hard GraphRecursionError.

Surfacing it (optional, follow-up)

  • Expose the current tool budget on /chat/model (src/tpk/server.py:156) alongside provider/model, so the UI can display it.
  • A management-UI control to set it at runtime is a separate follow-up (this issue is the env-var config; UI can come later, and would likely live wherever roles/corpus are managed).

Out of scope

  • Per-user or per-role budgets (single global budget for now).
  • Runtime UI editing (follow-up).

Tests (tests/test_agent.py / tests/test_server.py)

  • system_prompt() reflects TPK_AGENT_TOOL_BUDGET (the number appears in the prompt).
  • /chat uses TPK_AGENT_RECURSION_LIMIT for the LangGraph recursion_limit.
  • Defaults (12 / 40) apply when the env vars are unset.
  • Invalid values fall back to defaults rather than raising.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions