Avoid PytestAssertRewriteWarning in tests using runpytest

[pctablet505]

Fixes #1334

Pass --assert=plain to the in-process runpytest calls for the six tests that emit PytestAssertRewriteWarning, keeping the #1227 in-process fix intact. Also drop the explicit pytest_plugins assignment in the two event-loop fixture tests so they do not mark the already-loaded plugin for assertion rewriting.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.50%. Comparing base (1975f90) to head (c4a6ed4).
⚠️ Report is 11 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1508   +/-   ##
=======================================
  Coverage   94.50%   94.50%           
=======================================
  Files           2        2           
  Lines         510      510           
  Branches       62       62           
=======================================
  Hits          482      482           
  Misses         22       22           
  Partials        6        6           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[seifertm]

Thanks for the initiative.

The code changes do make sense superficially, but I'm reluctant to integrate them when there's no proof the patch actually solves the problem. Personally, I also don't understand the nature of what causes the issue. In order to progress this, I think we need to find answers to one or more of the following questions:

What is the reason assertion rewriting or setting pytest_plugins causes problems in the first place?
Can you provide a reproducer that verifies this patch fixes the linked GUIX issue?
Can you provide a reproducer that verifies for FreeBSD that this patch does not regress #1227 ?

[pctablet505]

You were right to hold off — the patch as I submitted it only fixed 4 of the 6 tests. Pushed a follow-up. Evidence below.

Cause

The give-away is in the Guix log: the module pytest names is tests, not pytest_asyncio, and the frame is _mark_plugins_for_rewrite, not import_plugin:

_pytest/config/__init__.py:1303: PytestAssertRewriteWarning: Module already imported so cannot be rewritten; tests
    self._mark_plugins_for_rewrite(hook, disable_autoload)

So pytest_plugins was never the trigger. What happens is that _mark_plugins_for_rewrite walks every distribution declaring a pytest11 entry point and feeds its file list to _iter_rewritable_modules, which turns any <dir>/__init__.py into the bare name <dir>. Our wheel is scoped to packages = ["pytest_asyncio"], but the sdist ships a pre-generated pytest_asyncio.egg-info/ whose SOURCES.txt lists the whole tree including tests/__init__.py — and importlib.metadata treats that egg-info as a distribution. Running pytest's own function on the two real metadata files:

$ python -c "
from _pytest.config import _iter_rewritable_modules
lines = [l.strip() for l in open('pytest_asyncio.egg-info/SOURCES.txt') if l.strip()]
print('sdist egg-info SOURCES.txt ->', sorted(set(_iter_rewritable_modules(lines))))
import zipfile
zf = zipfile.ZipFile('dist/pytest_asyncio-1.3.0-py3-none-any.whl')
rec = [l.split(',')[0] for l in zf.read('pytest_asyncio-1.3.0.dist-info/RECORD').decode().splitlines() if l]
print('installed wheel RECORD  ->', sorted(set(_iter_rewritable_modules(rec))))
"
sdist egg-info SOURCES.txt -> ['pytest_asyncio', 'tests']
installed wheel RECORD  -> ['pytest_asyncio']

Then mark_rewrite("tests") warns because tests is already in sys.modules under a plain loader — rewriting can't touch an executed module.

Why CI never sees it: at config time the build dir isn't on sys.path yet, so the egg-info isn't discovered and tests isn't marked; collection then puts the build dir on sys.path and imports tests with an ordinary loader; the in-process runpytest() shares sys.modules, now finds the egg-info, and marks a package that's already imported. python -m pytest puts the CWD on sys.path at startup, so the first step succeeds and nothing warns — Guix runs the console script, which doesn't. That one difference flips it.

Reproducer

uv venv venv --python 3.11
uv pip install --python ./venv/bin/python "pytest==9.0.2" hypothesis setuptools \
    setuptools_scm wheel "typing-extensions>=4.12" "backports-asyncio-runner>=1.1,<2"

curl -sLO https://files.pythonhosted.org/packages/source/p/pytest-asyncio/pytest_asyncio-1.3.0.tar.gz
tar xzf pytest_asyncio-1.3.0.tar.gz && cd pytest_asyncio-1.3.0

# build in-tree, like Guix's build phase — this re-reads the shipped SOURCES.txt
../venv/bin/python -c "from setuptools.build_meta import build_wheel; build_wheel('dist')"
uv pip install --python ../venv/bin/python --no-deps dist/*.whl

../venv/bin/pytest          #  -> 6 failed, 234 passed
../venv/bin/python -m pytest #  -> 240 passed

6 failed, 234 passed — the same 6 tests as the Guix log, out of the same 240, with the same warning text and frame. Same tree under -m passes.

Which half of the patch does the work

Five variants, clean restore of the pristine sdist each time, full suite:

variant	result
unpatched	6 failed, 234 passed
this PR as submitted	2 failed, 238 passed
--assert=plain on all 6, keep pytest_plugins	240 passed
--assert=plain on all 6, drop pytest_plugins	240 passed
only drop pytest_plugins	6 failed, 234 passed

--assert=plain is the whole fix and it needs to be on all six — I'd only put it on four. Dropping pytest_plugins = 'pytest_asyncio' does nothing either way, since the plugin is autoloaded through its entry point; that's also the answer to @McSinyx's question about what removing it does. Happy to drop that hunk if you'd rather keep the diff to --assert=plain only.

Worth noting this isn't a new idiom: 18afc9d — the commit that fixed #1227 — already added --assert=plain while converting runpytest_subprocess to runpytest, 17 times across test_asyncio_mark.py, modes/test_strict_mode.py, markers/test_invalid_arguments.py, test_fixture_loop_scopes.py and test_event_loop_fixture.py. It just didn't apply it to every test it converted — test_event_loop_fixture.py got it on one of its three tests and not the other two, which are two of the six failing here. This PR finishes that.

On #1227

That was a different failure — TypeError: fixture() got an unexpected keyword argument 'loop_scope', from runpytest_subprocess picking up the system's 0.23.8 — so it's purely about which pytest-asyncio the inner run imports. This PR doesn't touch that axis; the call inventory over tests/ is identical either side, in every file:

main: {'runpytest': 155, 'runpytest_subprocess': 6}
PR  : {'runpytest': 155, 'runpytest_subprocess': 6}

Checked empirically too, with 0.23.8 in a directory a subprocess finds on PYTHONPATH while the outer session holds the correct version — three pytester runs printing the inner plugin banner:

IN-PROCESS  (default)      : plugins: asyncio-1.3.0, hypothesis-6.161.6
IN-PROCESS  (--assert=plain): plugins: asyncio-1.3.0, hypothesis-6.161.6
SUBPROCESS  (default)      : plugins: asyncio-0.23.8, hypothesis-6.161.6

The subprocess reproduces #1227 on demand; --assert=plain leaves the in-process banner byte-identical, since it only controls whether the rewrite hook is installed, not import resolution. No FreeBSD hardware here, so that's the mechanism plus a Linux reproduction of the same shadowing, not a literal FreeBSD run.

Pushed

--assert=plain added to the two remaining tests in tests/test_event_loop_fixture.py, and a corrected changelog fragment (the old one blamed a system-installed copy, which isn't the mechanism). Verified against an sdist built from this branch: 2 failed, 296 passed → 298 passed.

One thing for you to decide: the more principled fix is probably packaging-side — not shipping tests/ paths in the sdist's SOURCES.txt, or not making the suite an importable tests package — which would cover every downstream distributor and any future runpytest() test at once. Bigger change and your call, so I kept this minimal; happy to take a run at it if you'd prefer.