Incompatibility with pytest-asyncio 1.0.0

[puddly]

Thank you for your wonderful project! We use it to magically speed up a lot of unit tests for https://github.com/zigpy/zha.

I noticed that we started having CI failures today and it looks like pytest-asyncio released version 1.0.0 and breaks compatibility with this plugin. I'm creating an issue just to keep track of this bug and in case someone has time to fix this bug. For now, we're keeping back pytest-asyncio.

[nolar]

Thanks for reporting. I will take a look soon (no promise though).

[nolar]

For the record, the cause of the issue seems to be the deprecation and removal of the event_loop fixture and replacing it with the event_loop_policty fixture, essentually a factory of event loops: https://github.com/pytest-dev/pytest-asyncio/releases/tag/v1.0.0

Looptime is expecting a specific fixture name "event_loop" when it injects itself in looptime/plugin.py:

@pytest.hookimpl(hookwrapper=True)  # type: ignore
def pytest_fixture_setup(fixturedef: Any, request: Any) -> Any:
    if fixturedef.argname == "event_loop":

Besides, it uses the event_loop in its own tests.

The fix seems to be easy.

[nolar]

… or not that easy.

The new pytest-asyncio uses a sophisticated approach with event loops of different scopes —namely session, package, module, class, function— and pre-creates the fixtures for event loops of those scopes from a single event loop policy, which seems to be session-scoped by default.

This, in turn, means that looptime cannot intercept the creation of event_loop_policy fixture (session-scoped) AND adequately match it with the individual test markers @pytest.mark.looptime(…) (function-scoped), in order to patch the event loop policy class.

Some kind of a function-scoped interceptor is needed, which requires diving into pytest-asyncio's internals to find a spot where this can be done. This is especially difficult since some tests might be using the higher-scoped fixtures of those pre-created event loops, so we cannot patch them, or we should patch-then-restore — that's tricky.

A quick look at the code shows that pytest-asyncio does not inject itself via interceptable fixtures or pytest hooks now, but rather via the wrappers (i.e. it wraps the async tests & fixtures and replaces them with these wrappers at runtime).

As the last resort, looptime can introduce its own pytest hook wrapper on the deepest function-scope level, check the current running loop class (regardless of the policy), patch it if it is not a looptime loop (as it did before), and un-patch it on exit. This looks somewhat dirty & hacky. Especially if there are fixtures above the function scope, which might be using the same event loop and prepare something in it, so they will fall out of the looptime optimization scope in that case. Maybe there is a better way.

As a rule of thumb, an event loop must be patched at creation, not in the middle of the run.

I need to research this new approach and think on a solution. This might take a time.

[nolar]

So, I have almost made a solution, which almost works — but hit a few conceptual issues. I will wait until they are clarified in pytest-dev/pytest-asyncio#1142 before continuing.

Long story short: this will require some breaking changes, since loop.time() cannot guarantee that the time starts with zero for every test anymore, because a single loop instance can be shared across many tests, and the time cannot be reset to zero (it will break its monotonicity). But I will make the looptime fixture simulate the old behaviour (or better say, keep it), so it measures the time from the test start, while loop.time() can return arbitrary values as usual. The codebases that use assert looptime == 123 will work as usual. Hopefully.

[nolar]

Released as looptime==0.3.

See #9 for the solution used, and the release notes of 0.3 for pontentially BREAKING changes (of an non-documented & non-promised behaviour, so I am not sure if "breaking" applies here).

---

If you want some extra behaviour with those higher-scoped event loops of pytest-asyncio>=1.0.0, I would appreciate a sample case explained & demoed with snippets — I could not imagine something realistic and worth the hassle (in pieces that were cut out). Related:

• Expand the looptime magic to multi-scoped non-running loops #11
• Patch & tweak multiple simultaneous event loops of pytest-asyncio>=1.0.0 #10