[Python] Test regressions with pytest-8.4.0: `tests/pytest/test_fixtures.py::test_given_fails_if_already_decorated_with_fixture` and `tests/pytest/test_capture.py::test_healthcheck_traceback_is_hidden`

[mgorny]

When running the test suite against pytest 8.4.0, I'm seeing two new test failures:

FAILED tests/pytest/test_fixtures.py::test_given_fails_if_already_decorated_with_fixture - AssertionError: assert {'errors': 1,...pped': 0, ...} == {'errors': 0,...pped': 0, ...}
FAILED tests/pytest/test_capture.py::test_healthcheck_traceback_is_hidden - assert (21 - 11) == 9

I've been able to reproduce on 8066974, with Python 3.13.4.

$ uv pip list
Package          Version Editable project location
---------------- ------- ---------------------------------
attrs            25.3.0
execnet          2.1.1
hypothesis       6.135.1 /tmp/hypothesis/hypothesis-python
iniconfig        2.1.0
packaging        25.0
pluggy           1.6.0
pygments         2.19.1
pytest           8.4.0
pytest-xdist     3.7.0
sortedcontainers 2.4.0

$ pytest tests/pytest/ -n12
========================================================= test session starts =========================================================
platform linux -- Python 3.13.4, pytest-8.4.0, pluggy-1.6.0
rootdir: /tmp/hypothesis
configfile: pytest.ini
plugins: xdist-3.7.0, hypothesis-6.135.1
12 workers [60 items]     
...................F.....................................F..                                                                    [100%]
============================================================== FAILURES ===============================================================
_________________________________________ test_given_fails_if_already_decorated_with_fixture __________________________________________
[gw10] linux -- Python 3.13.4 /tmp/hypothesis/hypothesis-python/.venv/bin/python3
Traceback (most recent call last):
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/_pytest/runner.py", line 344, in from_call
    result: TResult | None = func()
                             ~~~~^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/_pytest/runner.py", line 246, in <lambda>
    lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/_pytest/logging.py", line 850, in pytest_runtest_call
    yield
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/_pytest/skipping.py", line 263, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/_pytest/runner.py", line 178, in pytest_runtest_call
    item.runtest()
    ~~~~~~~~~~~~^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/_pytest/python.py", line 1670, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/_pytest/python.py", line 156, in pytest_pyfunc_call
    result = testfunction(**testargs)
  File "/tmp/hypothesis/hypothesis-python/tests/pytest/test_fixtures.py", line 193, in test_given_fails_if_already_decorated_with_fixture
    testdir.runpytest(script).assert_outcomes(failed=1)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/_pytest/pytester.py", line 613, in assert_outcomes
    assert_outcomes(
    ~~~~~~~~~~~~~~~^
        outcomes,
        ^^^^^^^^^
    ...<7 lines>...
        deselected=deselected,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/_pytest/pytester_assertions.py", line 74, in assert_outcomes
    assert obtained == expected
AssertionError: assert {'errors': 1,...pped': 0, ...} == {'errors': 0,...pped': 0, ...}
  
  Omitting 4 identical items, use -vv to show
  Differing items:
  {'failed': 0} != {'failed': 1}
  {'errors': 1} != {'errors': 0}
  Use -v to get more diff
-------------------------------------------------------- Captured stdout call ---------------------------------------------------------
running: /tmp/hypothesis/hypothesis-python/.venv/bin/python3 -mpytest --basetemp=/tmp/pytest-of-mgorny/pytest-1/popen-gw10/test_given_fails_if_already_decorated_with_fixture0/runpytest-0 /tmp/pytest-of-mgorny/pytest-1/popen-gw10/test_given_fails_if_already_decorated_with_fixture0/test_given_fails_if_already_decorated_with_fixture.py --basetemp=/tmp/pytest-of-mgorny/pytest-1/popen-gw10/basetemp
     in: /tmp/pytest-of-mgorny/pytest-1/popen-gw10/test_given_fails_if_already_decorated_with_fixture0
========================================================= test session starts =========================================================
platform linux -- Python 3.13.4, pytest-8.4.0, pluggy-1.6.0
rootdir: /tmp/pytest-of-mgorny/pytest-1/popen-gw10/test_given_fails_if_already_decorated_with_fixture0
plugins: xdist-3.7.0, hypothesis-6.135.1
collected 0 items / 1 error

=============================================================== ERRORS ================================================================
_______________________________ ERROR collecting test_given_fails_if_already_decorated_with_fixture.py ________________________________
test_given_fails_if_already_decorated_with_fixture.py:4: in <module>
    @given(x=st.integers())
     ^^^^^^^^^^^^^^^^^^^^^^
/tmp/hypothesis/hypothesis-python/src/hypothesis/core.py:1830: in run_test_as_given
    @impersonate(test)
     ^^^^^^^^^^^^^^^^^
/tmp/hypothesis/hypothesis-python/src/hypothesis/internal/reflection.py:655: in accept
    co_filename=target.__code__.co_filename,
                ^^^^^^^^^^^^^^^
E   AttributeError: 'FixtureFunctionDefinition' object has no attribute '__code__'. Did you mean: '__call__'?
======================================================= short test summary info =======================================================
ERROR test_given_fails_if_already_decorated_with_fixture.py - AttributeError: 'FixtureFunctionDefinition' object has no attribute '_...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
========================================================== 1 error in 0.77s ===========================================================
________________________________________________ test_healthcheck_traceback_is_hidden _________________________________________________
[gw1] linux -- Python 3.13.4 /tmp/hypothesis/hypothesis-python/.venv/bin/python3
Traceback (most recent call last):
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/_pytest/runner.py", line 344, in from_call
    result: TResult | None = func()
                             ~~~~^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/_pytest/runner.py", line 246, in <lambda>
    lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/_pytest/logging.py", line 850, in pytest_runtest_call
    yield
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/_pytest/skipping.py", line 263, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/_pytest/runner.py", line 178, in pytest_runtest_call
    item.runtest()
    ~~~~~~~~~~~~^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/_pytest/python.py", line 1670, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/tmp/hypothesis/hypothesis-python/.venv/lib/python3.13/site-packages/_pytest/python.py", line 156, in pytest_pyfunc_call
    result = testfunction(**testargs)
  File "/tmp/hypothesis/hypothesis-python/tests/pytest/test_capture.py", line 104, in test_healthcheck_traceback_is_hidden
    assert timeout_line - def_line == 9
AssertionError: assert (21 - 11) == 9
-------------------------------------------------------- Captured stdout call ---------------------------------------------------------
running: /tmp/hypothesis/hypothesis-python/.venv/bin/python3 -mpytest --basetemp=/tmp/pytest-of-mgorny/pytest-1/popen-gw1/test_healthcheck_traceback_is_hidden0/runpytest-0 /tmp/pytest-of-mgorny/pytest-1/popen-gw1/test_healthcheck_traceback_is_hidden0/test_healthcheck_traceback_is_hidden.py --verbose --basetemp=/tmp/pytest-of-mgorny/pytest-1/popen-gw1/basetemp
     in: /tmp/pytest-of-mgorny/pytest-1/popen-gw1/test_healthcheck_traceback_is_hidden0
========================================================= test session starts =========================================================
platform linux -- Python 3.13.4, pytest-8.4.0, pluggy-1.6.0 -- /tmp/hypothesis/hypothesis-python/.venv/bin/python3
cachedir: .pytest_cache
hypothesis profile 'default'
rootdir: /tmp/pytest-of-mgorny/pytest-1/popen-gw1/test_healthcheck_traceback_is_hidden0
plugins: xdist-3.7.0, hypothesis-6.135.1
collecting ... collected 1 item

test_healthcheck_traceback_is_hidden.py::test_healthcheck_traceback_is_hidden FAILED                                            [100%]

============================================================== FAILURES ===============================================================
________________________________________________ test_healthcheck_traceback_is_hidden _________________________________________________

    @given(integers().map(lambda x: time.sleep(0.2)))
>   def test_healthcheck_traceback_is_hidden(x):
                   ^^^
E   hypothesis.errors.FailedHealthCheck: Data generation is extremely slow: Only produced 5 valid examples in 1.00 seconds (0 invalid ones and 0 exceeded maximum size). Try decreasing size of the data you're generating (with e.g. max_size or max_leaves parameters).
E         count | fraction |    slowest draws (seconds)
E     x |    5  |    100%  |   0.200,  0.200,  0.200,  0.200,  0.200
E   See https://hypothesis.readthedocs.io/en/latest/reference/api.html#hypothesis.HealthCheck for more information about this. If you want to disable just this health check, add HealthCheck.too_slow to the suppress_health_check settings for this test.

test_healthcheck_traceback_is_hidden.py:5: FailedHealthCheck
------------------------------------------------------------- Hypothesis --------------------------------------------------------------
You can add @seed(289299243189538297233530936621266478381) to this test or run pytest with --hypothesis-seed=289299243189538297233530936621266478381 to reproduce this failure.
======================================================= short test summary info =======================================================
FAILED test_healthcheck_traceback_is_hidden.py::test_healthcheck_traceback_is_hidden - hypothesis.errors.FailedHealthCheck: Data gen...
========================================================== 1 failed in 1.52s ==========================================================
======================================================== slowest 20 durations =========================================================
6.49s call     hypothesis-python/tests/pytest/test_capture.py::test_output_emitting_unicode
3.40s call     hypothesis-python/tests/pytest/test_reporting.py::test_runs_reporting_hook
3.23s call     hypothesis-python/tests/pytest/test_junit.py::test_outputs_valid_xunit2_with_xdist
3.17s call     hypothesis-python/tests/pytest/test_fixtures.py::test_suppress_fixture_health_check_via_profile
2.93s call     hypothesis-python/tests/pytest/test_seeding.py::test_runs_repeatably_when_seed_is_set[foo]
2.79s call     hypothesis-python/tests/pytest/test_seeding.py::test_runs_repeatably_when_seed_is_set[42]
2.74s call     hypothesis-python/tests/pytest/test_statistics.py::test_prints_statistics_given_option_under_xdist_with_junitxml
2.52s call     hypothesis-python/tests/pytest/test_seeding.py::test_repeats_healthcheck_when_following_seed_instruction
2.44s call     hypothesis-python/tests/pytest/test_statistics.py::test_prints_statistics_given_option_under_xdist
2.28s call     hypothesis-python/tests/pytest/test_statistics.py::test_does_not_run_statistics_by_default
2.21s call     hypothesis-python/tests/pytest/test_statistics.py::test_prints_statistics_for_stateful_tests
1.99s call     hypothesis-python/tests/pytest/test_statistics.py::test_prints_statistics_given_option_with_junitxml
1.98s call     hypothesis-python/tests/pytest/test_fixtures.py::test_suppress_health_check_function_scoped_fixture
1.95s call     hypothesis-python/tests/pytest/test_capture.py::test_healthcheck_traceback_is_hidden
1.91s call     hypothesis-python/tests/pytest/test_fixtures.py::test_given_fails_if_already_decorated_with_fixture
1.86s call     hypothesis-python/tests/pytest/test_seeding.py::test_runs_repeatably_when_seed_is_set[0]
1.84s call     hypothesis-python/tests/pytest/test_fixtures.py::test_given_plus_function_scoped_non_autouse_fixtures_are_deprecated
1.81s call     hypothesis-python/tests/pytest/test_fixtures.py::test_fixture_errors_if_already_decorated_with_given
1.80s call     hypothesis-python/tests/pytest/test_statistics.py::test_prints_statistics_given_option
1.79s call     hypothesis-python/tests/pytest/test_profiles.py::test_runs_reporting_hook_in_any_verbose_mode[--hypothesis-verbosity=verbose]
======================================================= short test summary info =======================================================
FAILED tests/pytest/test_fixtures.py::test_given_fails_if_already_decorated_with_fixture - AssertionError: assert {'errors': 1,...pped': 0, ...} == {'errors': 0,...pped': 0, ...}
FAILED tests/pytest/test_capture.py::test_healthcheck_traceback_is_hidden - assert (21 - 11) == 9
==================================================== 2 failed, 58 passed in 10.93s ====================================================

[Zac-HD]

pretty sure the upstream change was pytest-dev/pytest#12473.

• the first failure (this test) wants us to give a nice error message in this error case; that's something we need to do for compatibility with the new Pytest version.
• the second failure (this test) wants a short traceback and we're still doing that; I think just updating the assertion would be sufficient.