You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
AgentDetails.tool_declarations come back as dicts after a JSON round trip, breaking judge metrics on saved runs #7876
Describe the Bug: AgentDetails.tool_declarations is typed list[Any], so the genai_types.Tool objects in it don't survive JSON. Two ordinary flows round-trip through JSON:
The eval service guide's "Score a run again without rerunning it" flow (InferenceResult.model_validate_json).
After either one, every tool comes back as a plain dict. Consumers of the field expect Tool objects:
rubric_based_multi_turn_trajectory_quality_v1 reads tool.function_declarations and raises AttributeError: 'dict' object has no attribute 'function_declarations'. LocalEvalService catches that, so the metric silently becomes NOT_EVALUATED.
The other judges serialize the tools through get_tool_declarations_as_json_str. Their prompt now carries the dict's null fields: 785 characters instead of 224 for one tool, including 17 nulls. The same run is therefore judged on different input depending on whether it was saved first.
Steps to Reproduce:
Build an AppDetails whose agent has a genai_types.Tool in tool_declarations. Inference records this on every invocation.
Round-trip it with AppDetails.model_validate_json(app_details.model_dump_json()).
Inspect the type of tool_declarations[0], or score the round-tripped invocations with rubric_based_multi_turn_trajectory_quality_v1.
Expected Behavior:
The round-tripped tool_declarations equal the original Tool objects, and the judges see the same tools text as before saving.
Observed Behavior:
entry type after round trip: dict
judge tools text: original 224 chars, restored 785 chars, nulls in restored: 17
File ".../rubric_based_multi_turn_trajectory_evaluator.py", line 270, in _assemble_dialogue_history
for func in tool.function_declarations or []:
AttributeError: 'dict' object has no attribute 'function_declarations'
restored run -> rubric_based_multi_turn_trajectory_quality_v1 NOT_EVALUATED score None
Environment Details:
ADK Library Version (pip show google-adk): 2.13.0 (also on main at e9e8f61)
Desktop OS: macOS
Python Version (python -V): 3.11.16
Model Information:
Are you using LiteLLM: No
Which model is being used: any (the round trip doesn't involve a model)
🟡 Optional Information
Minimal Reproduction Code:
fromgoogle.genaiimporttypesfromgoogle.adk.evaluation.app_detailsimportAgentDetails, AppDetailstool=types.Tool(function_declarations=[
types.FunctionDeclaration(name="set_light", description="Turns a light on or off.")])
app=AppDetails(agent_details={"home": AgentDetails(name="home", tool_declarations=[tool])})
restored=AppDetails.model_validate_json(app.model_dump_json())
print(type(restored.agent_details["home"].tool_declarations[0])) # <class 'dict'>
How often has this issue occurred?:
Always (100%), for any invocation whose agent has tools, once it has been through JSON
Activity
fix: load AgentDetails tool declarations back as genai Tools