Skip to content

Regression 0.14.0 -> 0.14.1 in combination with pytest-xdist: execnet.gateway_base.DumpError: can't serialize ... #202

@radoering

Description

@radoering

We noticed a regression between pytest-subtests 0.14.0 and 0.14.1 when using it in combination with pytest-xdist.

When executing the following snippet with pytest -n 1 test.py:

import unittest
from enum import StrEnum, auto

class MyEnum(StrEnum):
    A = auto()
    B = auto()

class MyUnitTest(unittest.TestCase):

    def test_enum_values(self):
        for value in (MyEnum.A, MyEnum.B):
            with self.subTest(value):
                assert True

it passes with pytest-subtests 0.14.0, but fails with pytest-subtests 0.14.1 with the following error:

error
==================================================================================================================================================== FAILURES ===================================================================================================================================================== 
___________________________________________________________________________________________________________________________________________ MyUnitTest.test_enum_values ___________________________________________________________________________________________________________________________________________ 
[gw0] win32 -- Python 3.12.10 C:\Users\randy\AppData\Local\pypoetry\Cache\virtualenvs\pytest-subtest-test-yf4kPUW1-py3.12\Scripts\python.exe

self = <test.MyUnitTest testMethod=test_enum_values>

    def test_enum_values(self):
        for value in (MyEnum.A, MyEnum.B):
>           with self.subTest(value):
                 ^^^^^^^^^^^^^^^^^^^

test.py:12:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
C:\Python312\Lib\contextlib.py:144: in __exit__
    next(self.gen)
C:\Python312\Lib\contextlib.py:144: in __exit__
    next(self.gen)
C:\Users\randy\AppData\Local\pypoetry\Cache\virtualenvs\pytest-subtest-test-yf4kPUW1-py3.12\Lib\site-packages\pytest_subtests\plugin.py:140: in _addSubTest
    self.ihook.pytest_runtest_logreport(report=sub_report)
C:\Users\randy\AppData\Local\pypoetry\Cache\virtualenvs\pytest-subtest-test-yf4kPUW1-py3.12\Lib\site-packages\pluggy\_hooks.py:512: in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
C:\Users\randy\AppData\Local\pypoetry\Cache\virtualenvs\pytest-subtest-test-yf4kPUW1-py3.12\Lib\site-packages\pluggy\_manager.py:120: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
C:\Users\randy\AppData\Local\pypoetry\Cache\virtualenvs\pytest-subtest-test-yf4kPUW1-py3.12\Lib\site-packages\xdist\remote.py:289: in pytest_runtest_logreport
    self.sendevent("testreport", data=data)
C:\Users\randy\AppData\Local\pypoetry\Cache\virtualenvs\pytest-subtest-test-yf4kPUW1-py3.12\Lib\site-packages\xdist\remote.py:126: in sendevent
    self.channel.send((name, kwargs))
C:\Users\randy\AppData\Local\pypoetry\Cache\virtualenvs\pytest-subtest-test-yf4kPUW1-py3.12\Lib\site-packages\execnet\gateway_base.py:912: in send
    self.gateway._send(Message.CHANNEL_DATA, self.id, dumps_internal(item))
                                                      ^^^^^^^^^^^^^^^^^^^^
C:\Users\randy\AppData\Local\pypoetry\Cache\virtualenvs\pytest-subtest-test-yf4kPUW1-py3.12\Lib\site-packages\execnet\gateway_base.py:1629: in dumps_internal
    return _Serializer().save(obj)  # type: ignore[return-value]
           ^^^^^^^^^^^^^^^^^^^^^^^
C:\Users\randy\AppData\Local\pypoetry\Cache\virtualenvs\pytest-subtest-test-yf4kPUW1-py3.12\Lib\site-packages\execnet\gateway_base.py:1647: in save
    self._save(obj)
C:\Users\randy\AppData\Local\pypoetry\Cache\virtualenvs\pytest-subtest-test-yf4kPUW1-py3.12\Lib\site-packages\execnet\gateway_base.py:1667: in _save
    dispatch(self, obj)
C:\Users\randy\AppData\Local\pypoetry\Cache\virtualenvs\pytest-subtest-test-yf4kPUW1-py3.12\Lib\site-packages\execnet\gateway_base.py:1744: in save_tuple
    self._save(item)
C:\Users\randy\AppData\Local\pypoetry\Cache\virtualenvs\pytest-subtest-test-yf4kPUW1-py3.12\Lib\site-packages\execnet\gateway_base.py:1667: in _save
    dispatch(self, obj)
C:\Users\randy\AppData\Local\pypoetry\Cache\virtualenvs\pytest-subtest-test-yf4kPUW1-py3.12\Lib\site-packages\execnet\gateway_base.py:1740: in save_dict
    self._write_setitem(key, value)
C:\Users\randy\AppData\Local\pypoetry\Cache\virtualenvs\pytest-subtest-test-yf4kPUW1-py3.12\Lib\site-packages\execnet\gateway_base.py:1734: in _write_setitem
    self._save(value)
C:\Users\randy\AppData\Local\pypoetry\Cache\virtualenvs\pytest-subtest-test-yf4kPUW1-py3.12\Lib\site-packages\execnet\gateway_base.py:1667: in _save
    dispatch(self, obj)
C:\Users\randy\AppData\Local\pypoetry\Cache\virtualenvs\pytest-subtest-test-yf4kPUW1-py3.12\Lib\site-packages\execnet\gateway_base.py:1740: in save_dict
    self._write_setitem(key, value)
C:\Users\randy\AppData\Local\pypoetry\Cache\virtualenvs\pytest-subtest-test-yf4kPUW1-py3.12\Lib\site-packages\execnet\gateway_base.py:1734: in _write_setitem
    self._save(value)
C:\Users\randy\AppData\Local\pypoetry\Cache\virtualenvs\pytest-subtest-test-yf4kPUW1-py3.12\Lib\site-packages\execnet\gateway_base.py:1667: in _save
    dispatch(self, obj)
C:\Users\randy\AppData\Local\pypoetry\Cache\virtualenvs\pytest-subtest-test-yf4kPUW1-py3.12\Lib\site-packages\execnet\gateway_base.py:1740: in save_dict
    self._write_setitem(key, value)
C:\Users\randy\AppData\Local\pypoetry\Cache\virtualenvs\pytest-subtest-test-yf4kPUW1-py3.12\Lib\site-packages\execnet\gateway_base.py:1734: in _write_setitem
    self._save(value)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <execnet.gateway_base._Serializer object at 0x0000015DD841B440>, obj = <MyEnum.A: 'a'>

    def _save(self, obj: object) -> None:
        tp = type(obj)
        try:
            dispatch = self._dispatch[tp]
        except KeyError:
            methodname = "save_" + tp.__name__
            meth: Callable[[_Serializer, object], None] | None = getattr(
                self.__class__, methodname, None
            )
            if meth is None:
>               raise DumpError(f"can't serialize {tp}") from None
E               execnet.gateway_base.DumpError: can't serialize <enum 'MyEnum'>

C:\Users\randy\AppData\Local\pypoetry\Cache\virtualenvs\pytest-subtest-test-yf4kPUW1-py3.12\Lib\site-packages\execnet\gateway_base.py:1665: DumpError
---------------------------------------------------------------------------------------------------------------------------------------------- Captured stdout call ----------------------------------------------------------------------------------------------------------------------------------------------- 
,
============================================================================================================================================= short test summary info ============================================================================================================================================= 
FAILED test.py::MyUnitTest::test_enum_values - execnet.gateway_base.DumpError: can't serialize <enum 'MyEnum'>
================================================================================================================================================ 1 failed in 0.80s ================================================================================================================================================

Executing the tests without pytest-xdist (pytest test.py) still succeeds.

The error can also be reproduced with kwargs, e.g. with self.subTest(path=Path('.')).

Workaround: Cast all arguments of subTest() to str.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions