Skip to content

fix(runners): classify plugin output before persisting events - #7185

Open
jaywang172 wants to merge 1 commit into
google:mainfrom
jaywang172:fix/persist-plugin-transformed-events
Open

jaywang172 wants to merge 1 commit into
google:mainfrom
jaywang172:fix/persist-plugin-transformed-events

Conversation

@jaywang172

@jaywang172 jaywang172 commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Link to Issue or Description of Change

Closes #7184. Related context: #3990 and #5161.

Problem:
When on_event_callback returns a replacement Event, Runner yields and appends that output but still evaluates persistence eligibility against the original Event. A replacement that changes partial=True to False reaches the caller as final while its event and state delta are missing from the session. Live content replacements can also be incorrectly excluded or bypass the inline-media filter. Equivalent in-place callbacks behave differently.

Solution:
Base persistence eligibility on the effective post-callback Event in node event consumption and plugin execution, including before-run early exits. This preserves the existing callback contract and introduces no new API. It complements #3990's callback-before-persistence behavior; it does not add the separate post-persistence hook proposed in #5161.

Testing Plan

Unit Tests:

  • Added 20 regression cases covering mutation/replacement, partial promotion/demotion, async/SSE/live, both live media conversion directions, and before-run early exit.
  • Full supported-Python-version test suite passes locally.

Revalidated against main at 5bc9e8c9:

New regression cases with unmodified main source:
7 failed, 13 passed

New regression cases with fix:
20 passed

Related suites with fix:
259 passed, 1 skipped, 3 xfailed

Related-suite command (Python 3.11.9):

PYTHONPATH=src python -m pytest \
  tests/unittests/test_runners.py \
  tests/unittests/runners \
  tests/unittests/live/test__runner_utils.py \
  tests/unittests/plugins/test_notification_error_callbacks.py \
  -q --disable-warnings

Changed-file pre-commit and git diff --check pass.

Manual End-to-End (E2E) Tests:

Ran the public Runner reproducer included in #7184 locally with a custom BaseAgent and real InMemorySessionService; no model/network calls. It emits a partial Event and compares in-place modification with replacement from on_event_callback.

                     before fix                     after fix
async / in-place     streamed=yes saved=yes state=yes  unchanged
async / replacement  streamed=yes saved=no  state=no   saved=yes state=yes
live  / in-place     streamed=yes saved=yes state=yes  unchanged
live  / replacement  streamed=yes saved=no  state=no   saved=yes state=yes

This is local Runner E2E, not a real model/backend live session.

Checklist

  • Read CONTRIBUTING.md.
  • Performed self-review of the scoped change.
  • Added regression tests that fail on unmodified main.
  • New regression tests and related suites pass locally.
  • Ran the public Runner reproducer end to end locally.
  • Changed-file pre-commit passes.
  • Full tox matrix for Python 3.10–3.14.
  • Mypy baseline comparison.

@jaywang172

Copy link
Copy Markdown
Contributor Author

Validation

Full local validation is complete.

Python Result
3.10 15,162 passed
3.11 15,171 passed
3.12 15,162 passed
3.13 15,162 passed
3.14 15,162 passed

Python 3.14 additionally reported 87 skipped, 27 xfailed, and 2 non-strict xpassed tests; tox exited successfully.

Additional validation:

  • mypy: 0 new diagnostics compared with unmodified main (766 existing diagnostics on both)
  • pre-commit run --all-files: passed
  • git diff --check: passed
  • uv build: passed
  • Source distribution and wheel built successfully
  • Clean-venv installation of the built wheel: passed
  • Public Runner smoke test against the installed wheel: passed for:
    • run_async + in-place callback mutation
    • run_async + replacement Event
    • run_live + in-place callback mutation
    • run_live + replacement Event

All four wheel-installed smoke cases yielded the transformed final event, persisted it to the session, and applied its state delta.

The initial Python 3.14 run produced two environment-specific import-loading failures because the local interpreter preloaded sitecustomize. Recreating the tox environment with a clean managed Python 3.14 interpreter eliminated both failures without any repository code, test, or allowlist changes.

GitHub-hosted CI is currently action_required pending repository authorization and has not executed yet; no GitHub CI pass is claimed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Runner evaluates plugin replacement event persistence using the original event

2 participants