Support for Pytest 8.0 about pytest-asyncio

[seifertm]

Apparently, the compatibility issues weren't as big in v0.23.4 as they were initially in the v0.23 releases.

Thanks to the investigation by @mgorny and the subsequent PR submitted by @akeeman there's now a pre-release of pytest-asyncio with pytest 8 support.

Version v0.23.5a0 does not contain code changes compared to v0.23.4, apart from fixes to the type annotations. Unless there are any unexpected issues, I'll tag a proper patch release by the end of the week.

[seifertm]

Thanks. Unfortunately, pytest-asyncio doesn't properly limit the maximum pytest version.

Pytest 8 introduces changes to the collection phase. Since it's a bit more effort for pytest-asyncio to accommodate those, I plan to temporarily exclude pytest>=8 from the compatible versions in a (post?) release. This doesn't resolve the issue, but it will prevent users from installing incompatible versions of pytest and pytest-asyncio.

[dolfinus]

@seifertm Could you please release 0.23.4? pytest 8.0 was just released, and users got errors while using pytest-asyncio, see #763

[seifertm]

There were no reports related to issues of pytest-asyncio-0.23.5a0 and pytest 8. There area lot of people wanting to upgrade and there were no significant changes required for pytest 8 support. Therefore, I tagged v0.23.5.

Thanks!

[seifertm]

@dolfinus @4danmi pytest-asyncio-0.23.4 is released. It contains a proper upper bound for the pytest version, so that it doesn't get installed with pytest 8.

[4danmi]

@seifertm Can we please release an hofix?

[seifertm]

pytest-asyncio v0.23.4a0 adds a maximum pytest version (pytest<8) to install_requires in setup.cfg. This should solve the issues in your CI that tests pre-release versions.

What's still left is to add actual support for pytest 8.

[mgorny]

Thanks!

For the record, Gentoo's been patching the pytest-8 support in since 2024-02-06, and we've had no issues reported so far.

[blink1073]

Understood, thank you!

[blink1073]

Confirmed, thank you! https://github.com/ipython/ipykernel/actions/runs/7468436406/job/20323873110

[analog-cbarber]

@dolfinus @4danmi pytest-asyncio-0.23.4 is released. It contains a proper upper bound for the pytest version, so that it doesn't get installed with pytest 8.

Seems like that fix is breaking in the conda-forge feedstock, so 0.23.4 is not yet available on conda-forge:

https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=866103&view=logs&j=656edd35-690f-5c53-9ba3-09c10d0bea97&t=986b1512-c876-5f92-0d81-ba851554a0a3&l=776

I think that this restriction also has to be added to the conda-forge recipe.

[mgorny]

Are things really that bad, though? From a quick run of the test suite, I'm seeing two test failures but I've tested some revdeps already and they all worked without problems. Perhaps 8.0.0 final is more compatible than the RCs.

Test output

$ make test
coverage run --parallel-mode --omit */_version.py -m pytest
========================================================= test session starts =========================================================
platform linux -- Python 3.11.7, pytest-8.0.0, pluggy-1.3.0
rootdir: /tmp/pytest-asyncio
configfile: setup.cfg
testpaths: docs/source, tests
plugins: asyncio-0.23.5.dev10+ge92efad, hypothesis-6.96.2
asyncio: mode=Mode.AUTO
collected 166 items                                                                                                                   

docs/source/concepts_function_scope_example.py .                                                                                [  0%]
docs/source/concepts_module_scope_example.py ..                                                                                 [  1%]
docs/source/how-to-guides/class_scoped_loop_example.py ..                                                                       [  3%]
docs/source/how-to-guides/module_scoped_loop_example.py ..                                                                      [  4%]
docs/source/how-to-guides/multiple_loops_example.py ..                                                                          [  5%]
docs/source/reference/fixtures/event_loop_example.py .                                                                          [  6%]
docs/source/reference/fixtures/event_loop_policy_example.py .                                                                   [  6%]
docs/source/reference/fixtures/event_loop_policy_parametrized_example.py ..                                                     [  7%]
docs/source/reference/markers/class_scoped_loop_custom_policies_strict_mode_example.py ..                                       [  9%]
docs/source/reference/markers/class_scoped_loop_strict_mode_example.py ..                                                       [ 10%]
docs/source/reference/markers/class_scoped_loop_with_fixture_strict_mode_example.py .                                           [ 10%]
docs/source/reference/markers/function_scoped_loop_pytestmark_strict_mode_example.py .                                          [ 11%]
docs/source/reference/markers/function_scoped_loop_strict_mode_example.py .                                                     [ 12%]
docs/source/reference/markers/module_scoped_loop_strict_mode_example.py ...                                                     [ 13%]
tests/async_fixtures/test_async_fixtures.py ..                                                                                  [ 15%]
tests/async_fixtures/test_async_fixtures_scope.py .                                                                             [ 15%]
tests/async_fixtures/test_async_fixtures_with_finalizer.py ..                                                                   [ 16%]
tests/async_fixtures/test_async_gen_fixtures.py ...                                                                             [ 18%]
tests/async_fixtures/test_nested.py .                                                                                           [ 19%]
tests/async_fixtures/test_parametrized_loop.py .                                                                                [ 19%]
tests/hypothesis/test_base.py .......                                                                                           [ 24%]
tests/loop_fixture_scope/test_loop_fixture_scope.py ..                                                                          [ 25%]
tests/markers/test_class_scope.py ..........                                                                                    [ 31%]
tests/markers/test_function_scope.py ......                                                                                     [ 34%]
tests/markers/test_module_scope.py ..........                                                                                   [ 40%]
tests/markers/test_package_scope.py .........                                                                                   [ 46%]
tests/markers/test_session_scope.py F..........                                                                                 [ 53%]
tests/modes/test_auto_mode.py ......                                                                                            [ 56%]
tests/modes/test_strict_mode.py .....                                                                                           [ 59%]
tests/test_asyncio_fixture.py ......                                                                                            [ 63%]
tests/test_asyncio_mark.py .......                                                                                              [ 67%]
tests/test_dependent_fixtures.py ..                                                                                             [ 68%]
tests/test_doctest.py ..                                                                                                        [ 69%]
tests/test_event_loop_fixture.py .                                                                                              [ 70%]
tests/test_event_loop_fixture_finalizer.py ......                                                                               [ 74%]
tests/test_event_loop_fixture_override_deprecation.py ....                                                                      [ 76%]
tests/test_explicit_event_loop_fixture_request.py .......                                                                       [ 80%]
tests/test_import.py ...                                                                                                        [ 82%]
tests/test_is_async_test.py ..F.                                                                                                [ 84%]
tests/test_multiloop.py .                                                                                                       [ 85%]
tests/test_port_factories.py ......                                                                                             [ 89%]
tests/test_simple.py ..........                                                                                                 [ 95%]
tests/test_skips.py .......                                                                                                     [ 99%]
tests/test_subprocess.py .                                                                                                      [100%]

============================================================== FAILURES ===============================================================
_____________________________________ test_asyncio_mark_provides_session_scoped_loop_strict_mode ______________________________________
/tmp/pytest-asyncio/tests/markers/test_session_scope.py:65: in test_asyncio_mark_provides_session_scoped_loop_strict_mode
    result.assert_outcomes(passed=4)
E   AssertionError: assert {'errors': 0,...pped': 0, ...} == {'errors': 0,...pped': 0, ...}
E     
E     Omitting 4 identical items, use -vv to show
E     Differing items:
E     {'passed': 3} != {'passed': 4}
E     {'failed': 1} != {'failed': 0}
E     Use -v to get more diff
-------------------------------------------------------- Captured stdout call ---------------------------------------------------------
============================= test session starts ==============================
platform linux -- Python 3.11.7, pytest-8.0.0, pluggy-1.3.0
rootdir: /tmp/pytest-of-mgorny/pytest-12/test_asyncio_mark_provides_session_scoped_loop_strict_mode0
plugins: asyncio-0.23.5.dev10+ge92efad, hypothesis-6.96.2
asyncio: mode=Mode.STRICT
collected 4 items

subpkg/test_subpkg.py F                                                  [ 25%]
test_module_one.py .                                                     [ 50%]
test_module_two.py ..                                                    [100%]

=================================== FAILURES ===================================
______________________ test_subpackage_runs_in_same_loop _______________________

    async def test_subpackage_runs_in_same_loop():
>       assert asyncio.get_running_loop() is shared_module.loop
E       assert <_UnixSelectorEventLoop running=True closed=False debug=False> is None
E        +  where <_UnixSelectorEventLoop running=True closed=False debug=False> = <built-in function get_running_loop>()
E        +    where <built-in function get_running_loop> = asyncio.get_running_loop
E        +  and   None = shared_module.loop

subpkg/test_subpkg.py:9: AssertionError
=========================== short test summary info ============================
FAILED subpkg/test_subpkg.py::test_subpackage_runs_in_same_loop - assert <_Un...
========================= 1 failed, 3 passed in 0.04s ==========================
____________________________________ test_returns_false_for_unmarked_coroutine_item_in_strict_mode ____________________________________
/tmp/pytest-asyncio/tests/test_is_async_test.py:81: in test_returns_false_for_unmarked_coroutine_item_in_strict_mode
    result.assert_outcomes(skipped=1)
E   AssertionError: assert {'errors': 0,...pped': 0, ...} == {'errors': 0,...pped': 1, ...}
E     
E     Omitting 4 identical items, use -vv to show
E     Differing items:
E     {'skipped': 0} != {'skipped': 1}
E     {'failed': 1} != {'failed': 0}
E     Use -v to get more diff
-------------------------------------------------------- Captured stdout call ---------------------------------------------------------
============================= test session starts ==============================
platform linux -- Python 3.11.7, pytest-8.0.0, pluggy-1.3.0
rootdir: /tmp/pytest-of-mgorny/pytest-12/test_returns_false_for_unmarked_coroutine_item_in_strict_mode0
plugins: asyncio-0.23.5.dev10+ge92efad, hypothesis-6.96.2
asyncio: mode=Mode.STRICT
collected 1 item

test_returns_false_for_unmarked_coroutine_item_in_strict_mode.py F       [100%]

=================================== FAILURES ===================================
__________________________________ test_coro ___________________________________

cls = <class '_pytest.runner.CallInfo'>
func = <function call_runtest_hook.<locals>.<lambda> at 0x7fa402873740>
when = 'call'
reraise = (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>)

    @classmethod
    def from_call(
        cls,
        func: Callable[[], TResult],
        when: Literal["collect", "setup", "call", "teardown"],
        reraise: Optional[
            Union[Type[BaseException], Tuple[Type[BaseException], ...]]
        ] = None,
    ) -> "CallInfo[TResult]":
        """Call func, wrapping the result in a CallInfo.
    
        :param func:
            The function to call. Called without arguments.
        :param when:
            The phase in which the function is called.
        :param reraise:
            Exception or exceptions that shall propagate if raised by the
            function, instead of being wrapped in the CallInfo.
        """
        excinfo = None
        start = timing.time()
        precise_start = timing.perf_counter()
        try:
>           result: Optional[TResult] = func()

/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py:345: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py:266: in <lambda>
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/pluggy/_hooks.py:493: in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/pluggy/_manager.py:115: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/pluggy/_manager.py:457: in traced_hookexec
    return outcome.get_result()
/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/pluggy/_manager.py:454: in <lambda>
    lambda: oldcall(hook_name, hook_impls, caller_kwargs, firstresult)
/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/_pytest/threadexception.py:87: in pytest_runtest_call
    yield from thread_exception_runtest_hook()
/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/_pytest/threadexception.py:63: in thread_exception_runtest_hook
    yield
/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/_pytest/unraisableexception.py:90: in pytest_runtest_call
    yield from unraisable_exception_runtest_hook()
/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/_pytest/unraisableexception.py:65: in unraisable_exception_runtest_hook
    yield
/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/_pytest/logging.py:839: in pytest_runtest_call
    yield from self._runtest_for(item, "call")
/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/_pytest/logging.py:822: in _runtest_for
    yield
/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/_pytest/capture.py:882: in pytest_runtest_call
    return (yield)
/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/_pytest/skipping.py:257: in pytest_runtest_call
    return (yield)
/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py:181: in pytest_runtest_call
    raise e
/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py:173: in pytest_runtest_call
    item.runtest()
/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/_pytest/python.py:1836: in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/pluggy/_hooks.py:493: in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/pluggy/_manager.py:115: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/pluggy/_manager.py:457: in traced_hookexec
    return outcome.get_result()
/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/pluggy/_manager.py:454: in <lambda>
    lambda: oldcall(hook_name, hook_impls, caller_kwargs, firstresult)
/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/_pytest/python.py:190: in pytest_pyfunc_call
    async_warn_and_skip(pyfuncitem.nodeid)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

nodeid = 'test_returns_false_for_unmarked_coroutine_item_in_strict_mode.py::test_coro'

    def async_warn_and_skip(nodeid: str) -> None:
        msg = "async def functions are not natively supported and have been skipped.\n"
        msg += (
            "You need to install a suitable plugin for your async framework, for example:\n"
        )
        msg += "  - anyio\n"
        msg += "  - pytest-asyncio\n"
        msg += "  - pytest-tornasync\n"
        msg += "  - pytest-trio\n"
        msg += "  - pytest-twisted"
>       warnings.warn(PytestUnhandledCoroutineWarning(msg.format(nodeid)))
E       pytest.PytestUnhandledCoroutineWarning: async def functions are not natively supported and have been skipped.
E       You need to install a suitable plugin for your async framework, for example:
E         - anyio
E         - pytest-asyncio
E         - pytest-tornasync
E         - pytest-trio
E         - pytest-twisted

/tmp/pytest-asyncio/.tox/py311/lib/python3.11/site-packages/_pytest/python.py:182: PytestUnhandledCoroutineWarning
=========================== short test summary info ============================
FAILED test_returns_false_for_unmarked_coroutine_item_in_strict_mode.py::test_coro
============================== 1 failed in 0.37s ===============================
=================================================== 2 failed, 164 passed in 21.83s ====================================================
make: *** [Makefile:24: test] Error 1

[seifertm]

@analog-cbarber I saw that you already opened conda-forge/pytest-asyncio-feedstock#41 and it looks your issue has been resolved already.

The feedstock is the correct place to report these kind of issues. I'm happy to help from the pytest-asyncio side, but none of the pytest-asyncio developers maintain the conda feedstock.

[analog-cbarber]

@analog-cbarber I saw that you already opened conda-forge/pytest-asyncio-feedstock#41 and it looks your issue has been resolved already.

The feedstock is the correct place to report these kind of issues. I'm happy to help from the pytest-asyncio side, but none of the pytest-asyncio developers maintain the conda feedstock.

Probably a good idea for one of the pytest-asyncio maintainers to sign on to maintain the conda feedstock so you don't get surprised by this kind of thing. It usually is not that much work.