Python: evaluate autoSend expressions before emitting output - #8508
Conversation
Evaluate configured autoSend values against workflow state in function-tool and agent actions. Preserve output bindings, defaults, and resume behavior, and cover automatic output with local regression tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Evaluate configured autoSend values against workflow state in function-tool and agent actions. Preserve output bindings, defaults, and resume behavior, and cover automatic output with local regression tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🔵 Needs a closer look
State-dependent expression evaluation affects resumed workflows, and only focused tests were reported.
Pull request overview
Evaluates declarative output.autoSend expressions against workflow state before emitting tool or agent results.
Changes:
- Adds state-aware evaluation for direct, approval-resume, and external-loop execution.
- Adds regression tests for expressions, defaults, failures, and result preservation.
- Documents automatic-output behavior.
File summaries
| File | Description |
|---|---|
| python/packages/declarative/tests/test_graph_coverage.py | Covers agent and evaluation-error scenarios. |
| python/packages/declarative/tests/test_function_tool_executor.py | Covers tool execution and approval resumption. |
| python/packages/declarative/README.md | Documents autoSend semantics. |
| python/packages/declarative/agent_framework_declarative/_workflows/_executors_tools.py | Evaluates tool autoSend against current state. |
| python/packages/declarative/agent_framework_declarative/_workflows/_executors_agents.py | Evaluates agent autoSend against current state. |
Review details
- Files reviewed: 5/5 changed files
- Comments generated: 0
- Review effort level: Balanced
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
Code Coverage OverviewLanguages: Python Python / code-coverage/pythonThe overall line coverage in commit 2bdee60 in the Show a line coverage summary of the most covered files.
Updated |
There was a problem hiding this comment.
MAF Automated Review — Iteration 1
Result: Findings reported
Scope: full PR (1 commit(s)): 1aafa57842fd
Model: gpt-5.6-sol-fast
Overview
The change correctly routes autoSend through workflow-state evaluation before Boolean conversion and adds broad coverage for literals, state references, failures, result storage, and explicit output. The strongest guardrails are pre-invocation evaluation and tests that preserve tool results, agent history, approval payloads, and external-loop state. Two lifecycle gaps remain: rejected approvals can be blocked by an output expression that is irrelevant to rejection, and resumed agent loops do not reevaluate state-dependent expressions as the .NET implementation does.
Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
2 verified findings remained after source verification (2 medium) across 2 files. Details are attached to the affected lines below.
Affected areas: python/packages/declarative/agent_framework_declarative/_workflows/_executors_agents.py, python/packages/declarative/agent_framework_declarative/_workflows/_executors_tools.py
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Defer tool output expression evaluation until execution is approved, and reevaluate agent output expressions on resumed turns. Cover changed-state and evaluator-error cases without changing stored output bindings. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Remove the granular automatic-output section from the package README and document the behavior alongside the executor classes and approval handler. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Motivation & Context
Declarative function-tool and agent actions should evaluate configured
output.autoSendexpressions before deciding whether to emit output. Converting an unevaluated expression string directly tobooltreats every nonempty expression as true, including expressions whose result is false.Description & Review Guide
autoSendthrough the existing workflow-state expression evaluator immediately before tool execution and on each agent invocation, including resumed turns. Rejected tool calls complete without evaluating the output expression. Add regression coverage for literals, state references, defaults, expression failures and continuation, and document the setting.Focused test command, from
python:Related Issue
N/A. No linked public issue.
Contribution Checklist
breaking changelabel (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.