gh-127257: ssl: Raise OSError for ERR_LIB_SYS #127361

encukou · 2024-11-28T13:19:38Z

One way OpenSSL can report a failed syscall is SSL_ERROR_SYSCALL. We have that one covered.

But, another way is ERR_LIB_SYS. Apparently, it's being used more now, and causing issues on OpenSSL 3.4.0 on Arch buildbots.

From the ERR_raise manpage:

ERR_LIB_SYS

    This "library code" indicates that a system error is
    being reported.  In this case, the reason code given
    to `ERR_raise()` and `ERR_raise_data()` *must* be
    `errno(3)`.

This PR only handles ERR_LIB_SYS for the high-lever error types
SSL_ERROR_SYSCALL and SSL_ERROR_SSL, i.e., not the ones where
OpenSSL indicates it has some more information about the issue.

I believe it's correct to raise OSError rather that SSLError
in these cases. That makes this a backwards-incompatible change
(but one we still might want to backport as a bugfix).

The error on Arch with OpenSSL 3.4.0 is broken pipe (EPIPE) when a client reads after the server shuts down. On my Arch, raising OSError makes test_poplip pass.

Issue: [tests] test_poplib fails with "env changed" on Arch Linux with OpenSSL 3.4: [SYS] unknown error (_ssl.c:2634) #127257

From the ERR_raise manpage: ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`.

encukou · 2024-11-28T13:35:47Z

!buildbot Arch.Linux

bedevere-bot · 2024-11-28T13:35:49Z

🤖 New build scheduled with the buildbot fleet by @encukou for commit 0098897 🤖

The command will test the builders whose names match following regular expression: Arch.Linux

The builders matched are:

AMD64 Arch Linux Perf PR
AMD64 Arch Linux Asan PR
AMD64 Arch Linux VintageParser PR
AMD64 Arch Linux Valgrind PR
AMD64 Arch Linux Usan PR
AMD64 Arch Linux Usan Function PR
AMD64 Arch Linux Asan Debug PR
AMD64 Arch Linux TraceRefs PR

petermarko · 2024-11-28T20:07:53Z

This fixes #125936 for me.
Thanks!

python/cpython#127331 python/cpython#127361 (From OE-Core rev: e271e9cbf896f1fb97d56c426e4217a6d2105ea4) Signed-off-by: Peter Marko <[email protected]> Signed-off-by: Richard Purdie <[email protected]>

python/cpython#127331 python/cpython#127361 Signed-off-by: Peter Marko <[email protected]> Signed-off-by: Richard Purdie <[email protected]>

python/cpython#127331 python/cpython#127361 (From OE-Core rev: e5f3a1793e34fb4cd1e53ca60b67f9a9f084b7a6) Signed-off-by: Peter Marko <[email protected]> Signed-off-by: Richard Purdie <[email protected]>

python/cpython#127331 python/cpython#127361 Signed-off-by: Peter Marko <[email protected]> Signed-off-by: Richard Purdie <[email protected]>

python/cpython#127331 python/cpython#127361 (From OE-Core rev: e5f3a1793e34fb4cd1e53ca60b67f9a9f084b7a6) Signed-off-by: Peter Marko <peter.markosiemens.com> Signed-off-by: Richard Purdie <richard.purdielinuxfoundation.org>

encukou · 2024-12-04T09:05:23Z

ssl experts (@jackjansen, @tiran, @dstufft, @alex): Do you want to check the change?
I think this is the correct place, and the correct error to raise, but I'd appreciate someone with domain knowledge.

encukou · 2024-12-09T12:31:40Z

If there are no objections, I'll merge tomorrow to unblock the buildbots.

vstinner

The change looks good to me. I don't know how to test it in a reliable way. Let's see how it goes with EPIPE and OpenSSL 3.4.

miss-islington-app · 2024-12-11T08:18:57Z

Thanks @encukou for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

From the ERR_raise manpage: ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`. This PR only handles ERR_LIB_SYS for the high-lever error types SSL_ERROR_SYSCALL and SSL_ERROR_SSL, i.e., not the ones where OpenSSL indicates it has some more information about the issue. (cherry picked from commit f4b31ed) Co-authored-by: Petr Viktorin <[email protected]>

bedevere-app · 2024-12-11T08:19:13Z

GH-127812 is a backport of this pull request to the 3.13 branch.

…127812) gh-127257: ssl: Raise OSError for ERR_LIB_SYS (GH-127361) From the ERR_raise manpage: ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`. This PR only handles ERR_LIB_SYS for the high-lever error types SSL_ERROR_SYSCALL and SSL_ERROR_SSL, i.e., not the ones where OpenSSL indicates it has some more information about the issue. (cherry picked from commit f4b31ed) Co-authored-by: Petr Viktorin <[email protected]>

miss-islington-app · 2024-12-12T11:45:15Z

Thanks @encukou for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

miss-islington-app · 2024-12-13T11:03:26Z

Thanks @encukou for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

From the ERR_raise manpage: ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`. This PR only handles ERR_LIB_SYS for the high-lever error types SSL_ERROR_SYSCALL and SSL_ERROR_SSL, i.e., not the ones where OpenSSL indicates it has some more information about the issue. (cherry picked from commit f4b31ed) Co-authored-by: Petr Viktorin <[email protected]>

bedevere-app · 2024-12-13T11:03:39Z

GH-127905 is a backport of this pull request to the 3.12 branch.

…127905) gh-127257: ssl: Raise OSError for ERR_LIB_SYS (GH-127361) From the ERR_raise manpage: ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`. This PR only handles ERR_LIB_SYS for the high-lever error types SSL_ERROR_SYSCALL and SSL_ERROR_SSL, i.e., not the ones where OpenSSL indicates it has some more information about the issue. (cherry picked from commit f4b31ed) Co-authored-by: Petr Viktorin <[email protected]>

python/cpython#127331 python/cpython#127361 (From OE-Core rev: e5f3a1793e34fb4cd1e53ca60b67f9a9f084b7a6) Signed-off-by: Peter Marko <[email protected]> Signed-off-by: Richard Purdie <[email protected]>

From the ERR_raise manpage: ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`. This PR only handles ERR_LIB_SYS for the high-lever error types SSL_ERROR_SYSCALL and SSL_ERROR_SSL, i.e., not the ones where OpenSSL indicates it has some more information about the issue.

OpenSSL 3.4.0 returns `ERR_LIB_SYS` in some more situations than it used to. In the case exercised by `SingleTLSLayerTestCase.test_close_after_handshake`, python/cpython#127361 (also backported to the 3.12 and 3.13 branches) turns this into `BrokenPipeError`. It seems reasonable to treat this in the same way as `ConnectionAbortedError` and `ConnectionResetError`.

stratakis · 2025-04-01T10:13:32Z

We started seeing this on 3.11 and 3.10 in Fedora Rawhide which moved to OpenSSL 3.5 beta (due to unrelated to this issue ssl test failures). Would this be considered a security fix to backport for the security releases as more and more distros start moving to the latest OpenSSL?

…127361) (pythonGH-127905) pythongh-127257: ssl: Raise OSError for ERR_LIB_SYS (pythonGH-127361) From the ERR_raise manpage: ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`. This PR only handles ERR_LIB_SYS for the high-lever error types SSL_ERROR_SYSCALL and SSL_ERROR_SSL, i.e., not the ones where OpenSSL indicates it has some more information about the issue. (cherry picked from commit f4b31ed) Co-authored-by: Petr Viktorin <[email protected]> (cherry picked from commit 7f707fa)

vstinner · 2025-04-01T10:45:28Z

We started seeing this on 3.11 and 3.10 in Fedora Rawhide which moved to OpenSSL 3.5 beta (due to unrelated to this issue ssl test failures). Would this be considered a security fix to backport for the security releases as more and more distros start moving to the latest OpenSSL?

IMO we can make an exception for this change and backport it to 3.11 and 3.10. I prepared backports:

3.11: [3.11] gh-127257: ssl: Raise OSError for ERR_LIB_SYS (GH-127361) (GH-127905) #131970
3.10: [3.10] gh-127257: ssl: Raise OSError for ERR_LIB_SYS (GH-127361) (GH-127905) #131971

…127905) (GH-131970) gh-127257: ssl: Raise OSError for ERR_LIB_SYS (GH-127361) From the ERR_raise manpage: ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`. This PR only handles ERR_LIB_SYS for the high-lever error types SSL_ERROR_SYSCALL and SSL_ERROR_SSL, i.e., not the ones where OpenSSL indicates it has some more information about the issue. (cherry picked from commit f4b31ed) (cherry picked from commit 7f707fa) Co-authored-by: Petr Viktorin <[email protected]>

…127905) (GH-131971) gh-127257: ssl: Raise OSError for ERR_LIB_SYS (GH-127361) From the ERR_raise manpage: ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`. This PR only handles ERR_LIB_SYS for the high-lever error types SSL_ERROR_SYSCALL and SSL_ERROR_SSL, i.e., not the ones where OpenSSL indicates it has some more information about the issue. (cherry picked from commit f4b31ed) (cherry picked from commit 7f707fa) Co-authored-by: Petr Viktorin <[email protected]>

…pythonGH-127905) (pythonGH-131971) pythongh-127257: ssl: Raise OSError for ERR_LIB_SYS (pythonGH-127361) From the ERR_raise manpage: ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`. This PR only handles ERR_LIB_SYS for the high-lever error types SSL_ERROR_SYSCALL and SSL_ERROR_SSL, i.e., not the ones where OpenSSL indicates it has some more information about the issue. (cherry picked from commit f4b31ed) (cherry picked from commit 7f707fa) Co-authored-by: Petr Viktorin <[email protected]>

The patch resolves the flakiness of test_ftplib Backported from upstream 3.10+: python#127361 Co-authored-by: Petr Viktorin <[email protected]>

ssl: Raise OSError for ERR_LIB_SYS

11e0523

From the ERR_raise manpage: ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`.

encukou requested a review from gpshead November 28, 2024 13:19

bedevere-app bot mentioned this pull request Nov 28, 2024

[tests] test_poplib fails with "env changed" on Arch Linux with OpenSSL 3.4: [SYS] unknown error (_ssl.c:2634) #127257

Closed

bedevere-app bot added the awaiting core review label Nov 28, 2024

Add blurb

0098897

vstinner approved these changes Dec 9, 2024

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting core review labels Dec 9, 2024

encukou merged commit f4b31ed into python:main Dec 10, 2024
50 of 53 checks passed

encukou deleted the ssl-sys-error branch December 10, 2024 10:56

bedevere-app bot removed the awaiting merge label Dec 10, 2024

encukou added the needs backport to 3.13 bugs and security fixes label Dec 11, 2024

bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Dec 11, 2024

encukou added the needs backport to 3.12 only security fixes label Dec 12, 2024

petermarko mentioned this pull request Dec 12, 2024

openssl-3.4.0 compatibility #125936

Closed

encukou added needs backport to 3.12 only security fixes and removed needs backport to 3.12 only security fixes labels Dec 13, 2024

bedevere-app bot removed the needs backport to 3.12 only security fixes label Dec 13, 2024

cjwatson mentioned this pull request Jan 9, 2025

SingleTLSLayerTestCase: Catch BrokenPipeError urllib3/urllib3#3547

Merged

lblasc mentioned this pull request Feb 17, 2025

python310/python311: fix failing tests with openssl >= 3.4 NixOS/nixpkgs#382794

Merged

13 tasks

bdarnell mentioned this pull request Mar 7, 2025

BaseIOStream.read_bytes may raise incorrect exception type with OpenSSL 3.4.0 tornadoweb/tornado#3443

Closed

vstinner mentioned this pull request Apr 1, 2025

[3.11] gh-127257: ssl: Raise OSError for ERR_LIB_SYS (GH-127361) (GH-127905) #131970

Merged

vstinner mentioned this pull request Apr 1, 2025

[3.10] gh-127257: ssl: Raise OSError for ERR_LIB_SYS (GH-127361) (GH-127905) #131971

Merged

stratakis mentioned this pull request Apr 14, 2025

test_ssl: ConnectionHandler shuts down ThreadedEchoServer #126500

Closed

stratakis pushed a commit to stratakis/cpython that referenced this pull request Apr 16, 2025

00457: ssl: Raise OSError for ERR_LIB_SYS

04ae7c2

The patch resolves the flakiness of test_ftplib Backported from upstream 3.10+: python#127361 Co-authored-by: Petr Viktorin <[email protected]>

stratakis mentioned this pull request Apr 16, 2025

00457: ssl: Raise OSError for ERR_LIB_SYS fedora-python/cpython#115

Merged

pmgdeb mentioned this pull request Apr 22, 2025

Deal with BrokenPipeError since openssl 3.4.x. gevent/gevent#2103

Open

hrnciar pushed a commit to fedora-python/cpython that referenced this pull request Apr 23, 2025

00457: ssl: Raise OSError for ERR_LIB_SYS

6031114

The patch resolves the flakiness of test_ftplib Backported from upstream 3.10+: python#127361 Co-authored-by: Petr Viktorin <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-127257: ssl: Raise OSError for ERR_LIB_SYS #127361

gh-127257: ssl: Raise OSError for ERR_LIB_SYS #127361

encukou commented Nov 28, 2024 •

edited by bedevere-app bot

Loading

encukou commented Nov 28, 2024

bedevere-bot commented Nov 28, 2024

petermarko commented Nov 28, 2024

encukou commented Dec 4, 2024

encukou commented Dec 9, 2024

vstinner left a comment

miss-islington-app bot commented Dec 11, 2024

bedevere-app bot commented Dec 11, 2024

miss-islington-app bot commented Dec 12, 2024

miss-islington-app bot commented Dec 13, 2024

bedevere-app bot commented Dec 13, 2024

stratakis commented Apr 1, 2025

vstinner commented Apr 1, 2025

gh-127257: ssl: Raise OSError for ERR_LIB_SYS #127361

gh-127257: ssl: Raise OSError for ERR_LIB_SYS #127361

Conversation

encukou commented Nov 28, 2024 • edited by bedevere-app bot Loading

encukou commented Nov 28, 2024

bedevere-bot commented Nov 28, 2024

petermarko commented Nov 28, 2024

encukou commented Dec 4, 2024

encukou commented Dec 9, 2024

vstinner left a comment

Choose a reason for hiding this comment

miss-islington-app bot commented Dec 11, 2024

bedevere-app bot commented Dec 11, 2024

miss-islington-app bot commented Dec 12, 2024

miss-islington-app bot commented Dec 13, 2024

bedevere-app bot commented Dec 13, 2024

stratakis commented Apr 1, 2025

vstinner commented Apr 1, 2025

encukou commented Nov 28, 2024 •

edited by bedevere-app bot

Loading