Skip to content

PyPy3.11 7.3.19 serialization crashes #933

Closed
@mgorny

Description

@mgorny

I'm doing early testing for PyPy3.11 7.3.19, and I see that IPyParallel seems to have some serialization trouble. This could be a bug in PyPy, but since the code is relatively complex, I'd appreciate if you could take a look first and possibly give us some hint.

Reproduced via:

uv venv -p pypy3.11
uv pip install -e '.[test]'
uv run pytest ipyparallel/tests/test_serialize.py

(running the whole test suite produces a lot of similar failures but with much more verbose tracebacks.

Backtrace / pytest output
========================================================= test session starts =========================================================
platform linux -- Python 3.11.11[pypy-7.3.19-final], pytest-8.3.5, pluggy-1.5.0
rootdir: /tmp/ipyparallel
configfile: pyproject.toml
plugins: cov-6.0.0, asyncio-0.21.2
asyncio: mode=Mode.AUTO
collected 15 items                                                                                                                    

ipyparallel/tests/test_serialize.py ....ssss..F.F.s                                                                             [100%]

============================================================== FAILURES ===============================================================
_____________________________________________________________ test_tuple ______________________________________________________________

    def test_tuple():
        tup = (lambda x: x, 1)
        bufs = serialize_object(tup)
>       canned = pickle.loads(bufs[0])

ipyparallel/tests/test_serialize.py:180: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/pypy3.11/pickle.py:1785: in _loads
    return _Unpickler(file, fix_imports=fix_imports, buffers=buffers,
/usr/lib/pypy3.11/pickle.py:1226: in load
    dispatch[key[0]](self)
/usr/lib/pypy3.11/pickle.py:1603: in load_reduce
    stack[-1] = func(*args)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

args = (1, 0, 0, 1, 1, 83, ...)

    def code_ctor(*args):
>       return types.CodeType(*args)
E       TypeError: expected str, got int object

ipyparallel/serialize/codeutil.py:20: TypeError
-------------------------------------------------------- Captured stdout call ---------------------------------------------------------
(<ipyparallel.serialize.canning.CannedFunction object at 0x0000558dce371830>, 1)
______________________________________________________________ test_list ______________________________________________________________

    def test_list():
        lis = [lambda x: x, 1]
        bufs = serialize_object(lis)
>       canned = pickle.loads(bufs[0])

ipyparallel/tests/test_serialize.py:202: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/pypy3.11/pickle.py:1785: in _loads
    return _Unpickler(file, fix_imports=fix_imports, buffers=buffers,
/usr/lib/pypy3.11/pickle.py:1226: in load
    dispatch[key[0]](self)
/usr/lib/pypy3.11/pickle.py:1603: in load_reduce
    stack[-1] = func(*args)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

args = (1, 0, 0, 1, 1, 83, ...)

    def code_ctor(*args):
>       return types.CodeType(*args)
E       TypeError: expected str, got int object

ipyparallel/serialize/codeutil.py:20: TypeError
-------------------------------------------------------- Captured stdout call ---------------------------------------------------------
[<ipyparallel.serialize.canning.CannedFunction object at 0x0000558dcf513be8>, 1]
========================================================== warnings summary ===========================================================
.venv/lib/pypy3.11/site-packages/jupyter_client/connect.py:22
  /tmp/ipyparallel/.venv/lib/pypy3.11/site-packages/jupyter_client/connect.py:22: DeprecationWarning: Jupyter is migrating its paths to use standard platformdirs
  given by the platformdirs library.  To remove this warning and
  see the appropriate new directories, set the environment variable
  `JUPYTER_PLATFORM_DIRS=1` and then run `jupyter --paths`.
  The use of platformdirs will be the default in `jupyter_core` v6
    from jupyter_core.paths import jupyter_data_dir, jupyter_runtime_dir, secure_write

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================================================== slowest 10 durations =========================================================

(10 durations < 0.005s hidden.  Use -vv to show these durations.)
======================================================= short test summary info =======================================================
SKIPPED [1] ipyparallel/tests/test_serialize.py:81: could not import 'numpy': No module named 'numpy'
SKIPPED [1] ipyparallel/tests/test_serialize.py:97: could not import 'numpy': No module named 'numpy'
SKIPPED [1] ipyparallel/tests/test_serialize.py:116: could not import 'numpy': No module named 'numpy'
SKIPPED [1] ipyparallel/tests/test_serialize.py:134: could not import 'numpy': No module named 'numpy'
SKIPPED [1] ipyparallel/tests/test_serialize.py:227: could not import 'numpy': No module named 'numpy'
FAILED ipyparallel/tests/test_serialize.py::test_tuple - TypeError: expected str, got int object
FAILED ipyparallel/tests/test_serialize.py::test_list - TypeError: expected str, got int object
========================================== 2 failed, 8 passed, 5 skipped, 1 warning in 0.69s ==========================================
uv pip list
Package                 Version     Editable project location
----------------------- ----------- -------------------------
asttokens               3.0.0
cffi                    1.17.1
comm                    0.2.2
coverage                7.6.12
debugpy                 1.8.12
decorator               5.2.1
executing               2.2.0
iniconfig               2.0.0
ipykernel               6.29.5
ipyparallel             9.1.0.dev0  /tmp/ipyparallel
ipython                 9.0.1
ipython-pygments-lexers 1.1.1
jedi                    0.19.2
jupyter-client          8.6.3
jupyter-core            5.7.2
matplotlib-inline       0.1.7
nest-asyncio            1.6.0
packaging               24.2
parso                   0.8.4
pexpect                 4.9.0
platformdirs            4.3.6
pluggy                  1.5.0
prompt-toolkit          3.0.50
psutil                  7.0.0
ptyprocess              0.7.0
pure-eval               0.2.3
pycparser               2.22
pygments                2.19.1
pytest                  8.3.5
pytest-asyncio          0.21.2
pytest-cov              6.0.0
python-dateutil         2.9.0.post0
pyzmq                   26.2.1
six                     1.17.0
stack-data              0.6.3
testpath                0.6.0
tornado                 6.4.2
tqdm                    4.67.1
traitlets               5.14.3
typing-extensions       4.12.2
wcwidth                 0.2.13

Gentoo Linux amd64
PyPy3.11 7.3.19
Reproduced on 0d5488f

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions