Skip to content

AUDIT-15 — Medium — Duplicate-tool-call detector only catches consecutive repeats #72

Closed

Description

@getdownandcode

Summary

Severity: Medium
Area: Agent loop
Affected files: agentkit/core/agent.py:218-229
Suggested branch: fix/duplicate-tool-call-detector
Suggested PR title: fix(agent): detect non-consecutive tool call loops

This is finding AUDIT-15 of 18 from a consolidated audit of agentkit/. It is deliberately self-contained so it can be fixed on its own branch with its own PR.

Related findings: Independent of the other findings.


AUDIT-15 — Medium — Duplicate-tool-call detector only catches consecutive repeats

File — agentkit/core/agent.py:218-229

                                for call in response.tool_calls:
                                    args_json = json.dumps(call.arguments, sort_keys=True, default=str)
                                    sig = (call.name, args_json)
                                    if sig == last_tool_signature:
                                        consecutive_duplicate_count += 1
                                    else:
                                        last_tool_signature = sig
                                        consecutive_duplicate_count = 1

                                    if (
                                        consecutive_duplicate_count
                                        >= self.config.max_consecutive_duplicate_tool_calls
                                    ):

Why this is a bug

Only the immediately preceding signature is remembered. An alternating pattern A, B, A, B, … resets the counter every time, so the documented "halt run if identical tool calls are repeated" guard never fires. The run then burns all max_steps (each step costing tokens, latency and money) before being stopped by the step limit instead of by the loop detector.

Pass cases

  • A repeating non-consecutive pattern (e.g. calculator(1), calculator(2), calculator(1), calculator(2), …) is detected as a loop and terminates the run with a clear failure_reason before max_steps is reached.
  • The consecutive-duplicate case keeps working (existing tests).
  • A legitimate sequence of different tool calls is not flagged.
  • The detector's scope is documented (window size / repetition count) and configurable.


Definition of done (per AGENTS.md)

  • New/updated unit and integration tests covering the pass cases above, all passing (pytest).
  • ruff check ., ruff format --check . and mypy agentkit clean.
  • Docstrings updated where behaviour changed; docs/PROGRESS.md updated (status, PR link, commit count, notes).
  • A new ADR in docs/DECISIONS.md if an architectural decision was made.
  • Branch pushed and PR opened (never commit directly to main).

Baseline at the time of writing: pytest is green on main (all tests pass; tests/real/* skipped unless AGENTKIT_REAL_INFRA=1).

Activity

  1. getdownandcode commented on Oct 20, 2026

    @getdownandcode
    OwnerAuthor

    Audit index (18 separate issues, fix one by one)

    ID Issue Severity
    AUDIT-01 #58 Critical
    AUDIT-02 #59 Critical
    AUDIT-03 #60 High
    AUDIT-04 #61 High
    AUDIT-05 #62 High
    AUDIT-06 #63 High/Medium
    AUDIT-07 #64 Medium-High
    AUDIT-08 #65 Medium-High
    AUDIT-09 #66 Medium-High
    AUDIT-10 #67 Medium-High
    AUDIT-11 #68 Medium
    AUDIT-12 #69 Medium
    AUDIT-13 #70 Medium
    AUDIT-14 #71 Medium
    AUDIT-15 #72 Medium
    AUDIT-16 #73 Medium
    AUDIT-17 #74 Low
    AUDIT-18 #75 Low/Deployment

    Each issue is self-contained: current code, reproduction, pass cases, suggested branch and PR title. Reference the relevant one with Fixes #NN in your PR.

  2. HamzaKhanBUIC commented on Oct 20, 2026

    @HamzaKhanBUIC

    Opened PR #83 with the complete fix and regression test suite:

    • In �gentkit/core/agent.py, enhanced loop detection to inspect tool call signature history for repeating multi-call cycles (max_tool_cycle_length=5, configurable max_duplicate_tool_call_cycles).
    • In �gentkit/core/errors.py, extended DuplicateToolCallLoopError with cycle_length attribute while maintaining backwards compatibility.
    • Added comprehensive unit tests in ests/unit/test_agent_loop_detector.py covering alternating 2-tool cycles, 3-tool repeating sequences, and
      aise_on_failure=True behavior.
    • Verified clean across pytest,
      uff check,
      uff format --check, and mypy agentkit.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions