Skip to content

AUDIT-13 — Medium — Tool JSON schemas drop $defs (unresolvable $ref) #70

Closed

Description

@getdownandcode

Summary

Severity: Medium
Area: Tools / schema
Affected files: agentkit/tools/schema.py:63-78
Suggested branch: fix/tool-schema-defs
Suggested PR title: fix(tools): preserve $defs in generated tool schemas

This is finding AUDIT-13 of 18 from a consolidated audit of agentkit/. It is deliberately self-contained so it can be fixed on its own branch with its own PR.

Related findings: Independent of the other findings.


AUDIT-13 — Medium — Tool JSON schemas drop $defs (unresolvable $ref)

File — agentkit/tools/schema.py:63-78

    raw_json_schema = args_model.model_json_schema()
    properties: dict[str, Any] = raw_json_schema.get("properties", {})
    required: list[str] = raw_json_schema.get("required", [])

    parameters: dict[str, Any] = {
        "type": "object",
        "properties": properties,
    }
    if required:
        parameters["required"] = required

Why this is a bug

Pydantic emits nested models and enums as $ref: "#/$defs/…", with the definitions in a sibling $defs key. The schema builder keeps only type/properties/required, so the $defs section is dropped and the emitted schema contains unresolvable references, which providers reject or mis-handle.

Reproduced

tool param typed as an Enum or a nested BaseModel
-> parameters = {"type":"object","properties":{"mode":{"$ref":"#/$defs/Mode", ...}}}
-> no "$defs" key in the emitted schema

Pass cases

  • A tool whose parameters contain an Enum or a nested BaseModel produces a schema in which every $ref resolves inside the emitted document.
  • $defs (and additionalProperties, title, default handling) are preserved in the emitted parameters.
  • The emitted schema is valid JSON Schema: a test resolves all $refs (or validates it with a JSON-Schema validator) and asserts no dangling references.
  • Existing simple tools (calculator, read_file, http_fetch, sql_readonly, web_search) produce byte-identical schemas to today (no regression for the built-ins).


Definition of done (per AGENTS.md)

  • New/updated unit and integration tests covering the pass cases above, all passing (pytest).
  • ruff check ., ruff format --check . and mypy agentkit clean.
  • Docstrings updated where behaviour changed; docs/PROGRESS.md updated (status, PR link, commit count, notes).
  • A new ADR in docs/DECISIONS.md if an architectural decision was made.
  • Branch pushed and PR opened (never commit directly to main).

Baseline at the time of writing: pytest is green on main (all tests pass; tests/real/* skipped unless AGENTKIT_REAL_INFRA=1).

Activity

  1. getdownandcode commented on Oct 20, 2026

    @getdownandcode
    OwnerAuthor

    Audit index (18 separate issues, fix one by one)

    ID Issue Severity
    AUDIT-01 #58 Critical
    AUDIT-02 #59 Critical
    AUDIT-03 #60 High
    AUDIT-04 #61 High
    AUDIT-05 #62 High
    AUDIT-06 #63 High/Medium
    AUDIT-07 #64 Medium-High
    AUDIT-08 #65 Medium-High
    AUDIT-09 #66 Medium-High
    AUDIT-10 #67 Medium-High
    AUDIT-11 #68 Medium
    AUDIT-12 #69 Medium
    AUDIT-13 #70 Medium
    AUDIT-14 #71 Medium
    AUDIT-15 #72 Medium
    AUDIT-16 #73 Medium
    AUDIT-17 #74 Low
    AUDIT-18 #75 Low/Deployment

    Each issue is self-contained: current code, reproduction, pass cases, suggested branch and PR title. Reference the relevant one with Fixes #NN in your PR.

  2. HamzaKhanBUIC commented on Oct 20, 2026

    @HamzaKhanBUIC

    Opened PR #81 with the fix and regression test suite:

    • In �gentkit/tools/schema.py, preserved $defs and �dditionalProperties emitted by �rgs_model.model_json_schema() on the output parameters dictionary.
    • Verified that tools with Enum and nested BaseModel parameters emit valid JSON Schema where every $ref successfully resolves inside the emitted document.
    • Verified that existing built-in tools (calculator,
      ead_file, http_fetch, sql_readonly, web_search) produce byte-identical schemas without extra keys.
    • Added comprehensive unit tests and recursive pointer resolution validation in ests/unit/test_tool_schema.py.
    • Verified clean across pytest,
      uff check,
      uff format --check, and mypy agentkit.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions