gh-126554: correct detection of gcc for TestNullDlsym.test_null_dlsym

[petermarko]

In case gcc is not available, it will throw exception and test fails. So catch the exception to skip the test correctly.

======================================================================
ERROR: test_null_dlsym (test.test_ctypes.test_dlerror.TestNullDlsym.test_null_dlsym)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.12/test/test_ctypes/test_dlerror.py", line 61, in test_null_dlsym
    retcode = subprocess.call(["gcc", "--version"],
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/subprocess.py", line 391, in call
    with Popen(*popenargs, **kwargs) as p:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/subprocess.py", line 1028, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.12/subprocess.py", line 1963, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'gcc'

• Issue: ctypes: We do not correctly handle NULL dlsym() return values #126554

[ghost]

All commit authors signed the Contributor License Agreement.

[bedevere-app]

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

[picnixz]

Wait, how come the returncode is not catching this kind of issue. I remember we changed the subprocess call but I don't remember why we left it as is.

1. Can you also use the issue number we used for adding this test?
2. Can you narrow down the exception being caught instead of a bare except? or find a way to avoid a try-except and use another feature of subprocess if possible?

cc @encukou (I don't remember why we didn't catch this)

[petermarko]

The trace is actually from 3.12.9, but the subprocess call throws exception also in 3.13 and 3.14-rc although I didn't try to run the testsuite in those.
The test was introduced based on gh-126554 and backported to 3.12 in gh-127764.
I can also just target 3.12 branch if it's not a problem for newer branches.

I'll rewrite this to catch just FileNotFoundError

[petermarko]

Beautified by adding FileNotFoundError exception type

Meanwhile I retested also 3.13.2, so I guess it's valid for all releases.

======================================================================
ERROR: test_null_dlsym (test.test_ctypes.test_dlerror.TestNullDlsym.test_null_dlsym)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.13/test/test_ctypes/test_dlerror.py", line 61, in test_null_dlsym
    retcode = subprocess.call(["gcc", "--version"],
                              stdout=subprocess.DEVNULL,
                              stderr=subprocess.DEVNULL)
  File "/usr/lib/python3.13/subprocess.py", line 397, in call
    with Popen(*popenargs, **kwargs) as p:
         ~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/subprocess.py", line 1038, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                        pass_fds, cwd, env,
                        ^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
                        gid, gids, uid, umask,
                        ^^^^^^^^^^^^^^^^^^^^^^
                        start_new_session, process_group)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/subprocess.py", line 1974, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'gcc'

[picnixz]

Ok, actually there is a way to make it work without a try-except: We need to pass the shell=True parameter. However it appears that the following happens:

>>> subprocess.call(["gcc", "--version"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
0
>>> subprocess.call(["gcc", "--version"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, shell=True)
1

So, we need to be a bit more careful with the actual return code! So, instead I either indeed suggest using a FileNotFoundError guard. Note that using which could work but this utility is not present everywhere:

subprocess.call(["which", "abc"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) # == 1
subprocess.call(["which", "gcc"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) # == 0

So, let's use the guard.

[encukou]

There is shutil.which, but, catching the error is more appropriate here.

Perhaps it's time to add a test helper to find gcc/clang, and skip test if it's not available. (But that would be for a new PR.)

[miss-islington-app]

Thanks @petermarko for the PR, and @encukou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13.
🐍🍒⛏🤖

[bedevere-app]

GH-129944 is a backport of this pull request to the 3.13 branch.

[encukou]

Thank you for the report & fix, @petermarko!

[bedevere-app]

GH-129945 is a backport of this pull request to the 3.12 branch.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Debian root 3.13 has failed when building commit cef406b.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1441/builds/897) and take a look at the build logs.
4. Check if the failure is related to this commit (cef406b) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1441/builds/897

Failed tests:

• test_os

Failed subtests:

• test_timerfd_ns_select - test.test_os.TimerfdTests.test_timerfd_ns_select

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/root/buildarea/3.13.angelico-debian-amd64/build/Lib/test/test_os.py", line 4450, in test_timerfd_ns_select
    self.assertEqual(self.read_count_signaled(fd), 1)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 3 != 1


Traceback (most recent call last):
  File "/root/buildarea/3.13.angelico-debian-amd64/build/Lib/test/test_os.py", line 4450, in test_timerfd_ns_select
    self.assertEqual(self.read_count_signaled(fd), 1)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 4 != 1