Skip to content

Python: evaluate autoSend expressions before emitting output - #8508

Merged
Jose Alvarez (jpalvarezl) merged 5 commits into
mainfrom
jpalvarezl-publish-autosend-expressions
Sep 18, 2026
Merged

Jose Alvarez (jpalvarezl) merged 5 commits into
mainfrom
jpalvarezl-publish-autosend-expressions

Conversation

@jpalvarezl

@jpalvarezl Jose Alvarez (jpalvarezl) commented Sep 18, 2026

Copy link
Copy Markdown
Member

Motivation & Context

Declarative function-tool and agent actions should evaluate configured output.autoSend expressions before deciding whether to emit output. Converting an unevaluated expression string directly to bool treats every nonempty expression as true, including expressions whose result is false.

Description & Review Guide

  • What are the major changes? Evaluate autoSend through the existing workflow-state expression evaluator immediately before tool execution and on each agent invocation, including resumed turns. Rejected tool calls complete without evaluating the output expression. Add regression coverage for literals, state references, defaults, expression failures and continuation, and document the setting.
  • What is the impact of these changes? False-valued expressions suppress automatic output. Existing default/literal coercion, output bindings, result storage and explicit downstream output are preserved. Resumed agent turns use current state rather than an earlier output decision. The selected executor, approval and evaluator suite passes 465 tests on Python 3.11 with PowerFx 0.0.34. Changed-file Ruff, source Pyright, test mypy and repository pre-commit hooks pass. The full repository unit suite was not run locally.
  • What do you want reviewers to focus on? Expression evaluation before Boolean conversion, rejection completion without output evaluation, preservation of stored results, and current-state decisions on resumed execution.

Focused test command, from python:

uv run --no-sync --package agent-framework-declarative pytest packages\declarative\tests\test_function_tool_executor.py packages\declarative\tests\test_graph_coverage.py packages\declarative\tests\test_graph_executors.py packages\declarative\tests\test_declarative_approval_binding.py packages\declarative\tests\test_declarative_state_path_safety.py -o addopts= -q --tb=short

Related Issue

N/A. No linked public issue.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

Evaluate configured autoSend values against workflow state in function-tool and agent actions. Preserve output bindings, defaults, and resume behavior, and cover automatic output with local regression tests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Evaluate configured autoSend values against workflow state in function-tool and agent actions. Preserve output bindings, defaults, and resume behavior, and cover automatic output with local regression tests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

State-dependent expression evaluation affects resumed workflows, and only focused tests were reported.

Pull request overview

Evaluates declarative output.autoSend expressions against workflow state before emitting tool or agent results.

Changes:

  • Adds state-aware evaluation for direct, approval-resume, and external-loop execution.
  • Adds regression tests for expressions, defaults, failures, and result preservation.
  • Documents automatic-output behavior.
File summaries
File Description
python/packages/declarative/tests/test_graph_coverage.py Covers agent and evaluation-error scenarios.
python/packages/declarative/tests/test_function_tool_executor.py Covers tool execution and approval resumption.
python/packages/declarative/README.md Documents autoSend semantics.
python/packages/declarative/agent_framework_declarative/_workflows/_executors_tools.py Evaluates tool autoSend against current state.
python/packages/declarative/agent_framework_declarative/_workflows/_executors_agents.py Evaluates agent autoSend against current state.
Review details
  • Files reviewed: 5/5 changed files
  • Comments generated: 0
  • Review effort level: Balanced

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

@github-code-quality

github-code-quality Bot commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: Python

Python / code-coverage/python

The overall line coverage in commit 2bdee60 in the jpalvarezl-publish-a... branch is 91%. Line coverage data for the main branch is not yet available.

Show a line coverage summary of the most covered files.
File main jpalvarezl-publish-a... 2bdee60 +/-
packages/core/a...ework/_tools.py 96%
packages/core/a...work/_skills.py 95%
packages/openai..._chat_client.py 94%
packages/core/a.../_compaction.py 94%
packages/core/a...ework/_types.py 93%
packages/core/a...ork/_vectors.py 93%
packages/core/a...bservability.py 93%
packages/core/a...amework/_mcp.py 92%
packages/ag-ui/...i/_agent_run.py 89%
packages/core/a...ork/security.py 89%

Updated September 18, 2026 13:10 UTC

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAF Automated Review — Iteration 1

Result: Findings reported
Scope: full PR (1 commit(s)): 1aafa57842fd
Model: gpt-5.6-sol-fast

Overview

The change correctly routes autoSend through workflow-state evaluation before Boolean conversion and adds broad coverage for literals, state references, failures, result storage, and explicit output. The strongest guardrails are pre-invocation evaluation and tests that preserve tool results, agent history, approval payloads, and external-loop state. Two lifecycle gaps remain: rejected approvals can be blocked by an output expression that is irrelevant to rejection, and resumed agent loops do not reevaluate state-dependent expressions as the .NET implementation does.

Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
2 verified findings remained after source verification (2 medium) across 2 files. Details are attached to the affected lines below.

Affected areas: python/packages/declarative/agent_framework_declarative/_workflows/_executors_agents.py, python/packages/declarative/agent_framework_declarative/_workflows/_executors_tools.py

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Defer tool output expression evaluation until execution is approved, and reevaluate agent output expressions on resumed turns. Cover changed-state and evaluator-error cases without changing stored output bindings.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Remove the granular automatic-output section from the package README and document the behavior alongside the executor classes and approval handler.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@jpalvarezl
Jose Alvarez (jpalvarezl) added this pull request to the merge queue Sep 18, 2026
Merged via the queue into main with commit f99fcc7 Sep 18, 2026
45 checks passed
@jpalvarezl
Jose Alvarez (jpalvarezl) deleted the jpalvarezl-publish-autosend-expressions branch September 18, 2026 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants