Agent test UI: pick tools and agents rather than type them - #481
Open
yuyixg wants to merge 1 commit into
Open
Conversation
A tool name and an agent name are both things an author has in front of them and cannot spell from memory, and both fail silently when mistyped: a toolCalled on a function that does not exist can never pass, and a routedToAgent on a misspelt agent fails every run while reading like a real regression. The datalist on the tool fields only suggested -- it never stopped a typo. - toolCalled / toolNotCalled target, and a mock's function name, are now searchable pickers over the tool catalogue. - routedToAgent expected is an agent picker. It stores the name, not the id: both are accepted (AgentChainHop.Matches), and the name is what the run report prints in the expected and actual columns. - agentChain expected is an ordered chip list -- add from a picker, remove, move. Ordered rather than a multi-select because in `ordered` and `exact` mode the sequence IS the assertion, and a multi-select hands its options back in list order rather than click order. The move controls appear only under those two modes. Two things the pickers needed to be usable at all: The tool list is no longer just the suite agent's. A case can enter on a different agent, and the negative form of a routing assertion names the signature tool of the agent the conversation must stay away from -- by definition someone else's. Tools are now cached per agent id and unioned across the suite agent, the entry agent and the involved agents, kept current by one effect rather than a call at each of the five places those fields can change. Every picker keeps a manual escape hatch, and a stored value the picker cannot offer switches its own row to the text box instead of being quietly dropped on the next save. Neither catalogue is the whole truth: MCP tools are discovered at run time, and a stored case can name a function its agent has since lost. The manual-entry flags are editor-only -- buildPayload lists the fields the backend has, so they never reach the API. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A tool name and an agent name are both things an author has in front of them and cannot spell from memory, and both fail silently when mistyped: a toolCalled on a function that does not exist can never pass, and a routedToAgent on a misspelt agent fails every run while reading like a real regression. The datalist on the tool fields only suggested -- it never stopped a typo.
orderedandexactmode the sequence IS the assertion, and a multi-select hands its options back in list order rather than click order. The move controls appear only under those two modes.Two things the pickers needed to be usable at all:
The tool list is no longer just the suite agent's. A case can enter on a different agent, and the negative form of a routing assertion names the signature tool of the agent the conversation must stay away from -- by definition someone else's. Tools are now cached per agent id and unioned across the suite agent, the entry agent and the involved agents, kept current by one effect rather than a call at each of the five places those fields can change.
Every picker keeps a manual escape hatch, and a stored value the picker cannot offer switches its own row to the text box instead of being quietly dropped on the next save. Neither catalogue is the whole truth: MCP tools are discovered at run time, and a stored case can name a function its agent has since lost.
The manual-entry flags are editor-only -- buildPayload lists the fields the backend has, so they never reach the API.