Merge branch 'main' into fix-issue-135447

Author: Yzi-Li

.github/workflows/jit.yml

@@ -5,6 +5,8 @@ on:
       - '**jit**'
       - 'Python/bytecodes.c'
       - 'Python/optimizer*.c'
+      - 'Python/executor_cases.c.h'
+      - 'Python/optimizer_cases.c.h'
       - '!Python/perf_jit_trampoline.c'
       - '!**/*.md'
       - '!**/*.ini'
@@ -13,6 +15,8 @@ on:
       - '**jit**'
       - 'Python/bytecodes.c'
       - 'Python/optimizer*.c'
+      - 'Python/executor_cases.c.h'
+      - 'Python/optimizer_cases.c.h'
       - '!Python/perf_jit_trampoline.c'
       - '!**/*.md'
       - '!**/*.ini'

.github/workflows/posix-deps-apt.sh

@@ -25,3 +25,10 @@ apt-get -yq install \
     uuid-dev \
     xvfb \
     zlib1g-dev
+
+# Workaround missing libmpdec-dev on ubuntu 24.04:
+# https://launchpad.net/~ondrej/+archive/ubuntu/php
+# https://deb.sury.org/
+sudo add-apt-repository ppa:ondrej/php
+apt-get update
+apt-get -yq install libmpdec-dev

.pre-commit-config.yaml

@@ -34,6 +34,13 @@ repos:
         name: Run Black on Tools/jit/
         files: ^Tools/jit/
 
+  - repo: https://github.com/Lucas-C/pre-commit-hooks
+    rev: v1.5.5
+    hooks:
+      - id: remove-tabs
+        types: [python]
+        exclude: ^Tools/c-analyzer/cpython/_parser.py
+
   - repo: https://github.com/pre-commit/pre-commit-hooks
     rev: v5.0.0
     hooks:

Doc/c-api/arg.rst

@@ -686,6 +686,12 @@ Building values
    ``p`` (:class:`bool`) [int]
       Convert a C :c:expr:`int` to a Python :class:`bool` object.
 
+      Be aware that this format requires an ``int`` argument.
+      Unlike most other contexts in C, variadic arguments are not coerced to
+      a suitable type automatically.
+      You can convert another type (for example, a pointer or a float) to a
+      suitable ``int`` value using ``(x) ? 1 : 0`` or ``!!x``.
+
       .. versionadded:: 3.14
 
    ``c`` (:class:`bytes` of length 1) [char]

Doc/c-api/capsule.rst

@@ -105,9 +105,19 @@ Refer to :ref:`using-capsules` for more information on using these objects.
    ``module.attribute``.  The *name* stored in the capsule must match this
    string exactly.
 
+   This function splits *name* on the ``.`` character, and imports the first
+   element. It then processes further elements using attribute lookups.
+
    Return the capsule's internal *pointer* on success.  On failure, set an
    exception and return ``NULL``.
 
+   .. note::
+
+      If *name* points to an attribute of some submodule or subpackage, this
+      submodule or subpackage must be previously imported using other means
+      (for example, by using :c:func:`PyImport_ImportModule`) for the
+      attribute lookups to succeed.
+
    .. versionchanged:: 3.3
       *no_block* has no effect anymore.
 

Doc/c-api/function.rst

@@ -95,6 +95,13 @@ There are a few functions specific to Python functions.
 
    .. versionadded:: 3.12
 
+
+.. c:function:: PyObject* PyFunction_GetKwDefaults(PyObject *op)
+
+   Return the keyword-only argument default values of the function object *op*. This can be a
+   dictionary of arguments or ``NULL``.
+
+
 .. c:function:: PyObject* PyFunction_GetClosure(PyObject *op)
 
    Return the closure associated with the function object *op*. This can be ``NULL``
@@ -123,6 +130,19 @@ There are a few functions specific to Python functions.
    Raises :exc:`SystemError` and returns ``-1`` on failure.
 
 
+.. c:function:: PyObject *PyFunction_GET_CODE(PyObject *op)
+                PyObject *PyFunction_GET_GLOBALS(PyObject *op)
+                PyObject *PyFunction_GET_MODULE(PyObject *op)
+                PyObject *PyFunction_GET_DEFAULTS(PyObject *op)
+                PyObject *PyFunction_GET_KW_DEFAULTS(PyObject *op)
+                PyObject *PyFunction_GET_CLOSURE(PyObject *op)
+                PyObject *PyFunction_GET_ANNOTATIONS(PyObject *op)
+
+   These functions are similar to their ``PyFunction_Get*`` counterparts, but
+   do not do type checking. Passing anything other than an instance of
+   :c:data:`PyFunction_Type` is undefined behavior.
+
+
 .. c:function:: int PyFunction_AddWatcher(PyFunction_WatchCallback callback)
 
    Register *callback* as a function watcher for the current interpreter.

Doc/c-api/init.rst

@@ -1250,7 +1250,7 @@ All of the following functions must be called after :c:func:`Py_Initialize`.
 .. c:function:: void PyInterpreterState_Clear(PyInterpreterState *interp)
 
    Reset all information in an interpreter state object.  There must be
-   an :term:`attached thread state` for the the interpreter.
+   an :term:`attached thread state` for the interpreter.
 
    .. audit-event:: cpython.PyInterpreterState_Clear "" c.PyInterpreterState_Clear
 
@@ -2277,6 +2277,18 @@ The C-API provides a basic mutual exclusion lock.
 
    .. versionadded:: 3.13
 
+.. c:function:: int PyMutex_IsLocked(PyMutex *m)
+
+   Returns non-zero if the mutex *m* is currently locked, zero otherwise.
+
+   .. note::
+
+      This function is intended for use in assertions and debugging only and
+      should not be used to make concurrency control decisions, as the lock
+      state may change immediately after the check.
+
+   .. versionadded:: next
+
 .. _python-critical-section-api:
 
 Python Critical Section API

Doc/c-api/long.rst

@@ -439,7 +439,7 @@ distinguished from a number.  Use :c:func:`PyErr_Occurred` to disambiguate.
    All *n_bytes* of the buffer are written: large buffers are padded with
    zeroes.
 
-   If the returned value is greater than than *n_bytes*, the value was
+   If the returned value is greater than *n_bytes*, the value was
    truncated: as many of the lowest bits of the value as could fit are written,
    and the higher bits are ignored. This matches the typical behavior
    of a C-style downcast.

Doc/c-api/object.rst

@@ -197,6 +197,13 @@ Object Protocol
    in favour of using :c:func:`PyObject_DelAttr`, but there are currently no
    plans to remove it.
 
+   The function must not be called with ``NULL`` *v* and an an exception set.
+   This case can arise from forgetting ``NULL`` checks and would delete the
+   attribute.
+
+   .. versionchanged:: next
+      Must not be called with NULL value if an exception is set.
+
 
 .. c:function:: int PyObject_SetAttrString(PyObject *o, const char *attr_name, PyObject *v)
 
@@ -207,6 +214,10 @@ Object Protocol
    If *v* is ``NULL``, the attribute is deleted, but this feature is
    deprecated in favour of using :c:func:`PyObject_DelAttrString`.
 
+   The function must not be called with ``NULL`` *v* and an an exception set.
+   This case can arise from forgetting ``NULL`` checks and would delete the
+   attribute.
+
    The number of different attribute names passed to this function
    should be kept small, usually by using a statically allocated string
    as *attr_name*.
@@ -215,6 +226,10 @@ Object Protocol
    For more details, see :c:func:`PyUnicode_InternFromString`, which may be
    used internally to create a key object.
 
+   .. versionchanged:: next
+      Must not be called with NULL value if an exception is set.
+
+
 .. c:function:: int PyObject_GenericSetAttr(PyObject *o, PyObject *name, PyObject *value)
 
    Generic attribute setter and deleter function that is meant

Doc/data/refcounts.dat

@@ -963,21 +963,45 @@ PyFunction_Check:PyObject*:o:0:
 PyFunction_GetAnnotations:PyObject*::0:
 PyFunction_GetAnnotations:PyObject*:op:0:
 
+PyFunction_GET_ANNOTATIONS:PyObject*::0:
+PyFunction_GET_ANNOTATIONS:PyObject*:op:0:
+
 PyFunction_GetClosure:PyObject*::0:
 PyFunction_GetClosure:PyObject*:op:0:
 
+PyFunction_GET_CLOSURE:PyObject*::0:
+PyFunction_GET_CLOSURE:PyObject*:op:0:
+
 PyFunction_GetCode:PyObject*::0:
 PyFunction_GetCode:PyObject*:op:0:
 
+PyFunction_GET_CODE:PyObject*::0:
+PyFunction_GET_CODE:PyObject*:op:0:
+
 PyFunction_GetDefaults:PyObject*::0:
 PyFunction_GetDefaults:PyObject*:op:0:
 
+PyFunction_GET_DEFAULTS:PyObject*::0:
+PyFunction_GET_DEFAULTS:PyObject*:op:0:
+
+PyFunction_GetKwDefaults:PyObject*::0:
+PyFunction_GetKwDefaults:PyObject*:op:0:
+
+PyFunction_GET_KW_DEFAULTS:PyObject*::0:
+PyFunction_GET_KW_DEFAULTS:PyObject*:op:0:
+
 PyFunction_GetGlobals:PyObject*::0:
 PyFunction_GetGlobals:PyObject*:op:0:
 
+PyFunction_GET_GLOBALS:PyObject*::0:
+PyFunction_GET_GLOBALS:PyObject*:op:0:
+
 PyFunction_GetModule:PyObject*::0:
 PyFunction_GetModule:PyObject*:op:0:
 
+PyFunction_GET_MODULE:PyObject*::0:
+PyFunction_GET_MODULE:PyObject*:op:0:
+
 PyFunction_New:PyObject*::+1:
 PyFunction_New:PyObject*:code:+1:
 PyFunction_New:PyObject*:globals:+1:

Doc/deprecations/c-api-pending-removal-in-3.15.rst

@@ -1,7 +1,6 @@
 Pending removal in Python 3.15
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-* The bundled copy of ``libmpdecimal``.
 * The :c:func:`!PyImport_ImportModuleNoBlock`:
   Use :c:func:`PyImport_ImportModule` instead.
 * :c:func:`PyWeakref_GetObject` and :c:func:`PyWeakref_GET_OBJECT`:

Doc/deprecations/c-api-pending-removal-in-3.16.rst

@@ -0,0 +1,4 @@
+Pending removal in Python 3.16
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* The bundled copy of ``libmpdec``.

Doc/deprecations/pending-removal-in-3.15.rst

@@ -97,6 +97,8 @@ Pending removal in Python 3.15
     After eight years in the :mod:`typing` module,
     it has yet to be supported by any major type checker.
 
+* :mod:`!sre_compile`, :mod:`!sre_constants` and :mod:`!sre_parse` modules.
+
 * :mod:`wave`:
 
   * The ``getmark()``, ``setmark()`` and ``getmarkers()`` methods of

Doc/deprecations/pending-removal-in-future.rst

@@ -89,8 +89,6 @@ although there is currently no date scheduled for their removal.
   underscore.
   (Contributed by Serhiy Storchaka in :gh:`91760`.)
 
-* :mod:`!sre_compile`, :mod:`!sre_constants` and :mod:`!sre_parse` modules.
-
 * :mod:`shutil`: :func:`~shutil.rmtree`'s *onerror* parameter is deprecated in
   Python 3.12; use the *onexc* parameter instead.
 

Doc/extending/newtypes_tutorial.rst

@@ -277,7 +277,7 @@ be an instance of a subclass.
    The explicit cast to ``CustomObject *`` above is needed because we defined
    ``Custom_dealloc`` to take a ``PyObject *`` argument, as the ``tp_dealloc``
    function pointer expects to receive a ``PyObject *`` argument.
-   By assigning to the the ``tp_dealloc`` slot of a type, we declare
+   By assigning to the ``tp_dealloc`` slot of a type, we declare
    that it can only be called with instances of our ``CustomObject``
    class, so the cast to ``(CustomObject *)`` is safe.
    This is object-oriented polymorphism, in C!

Doc/howto/functional.rst

@@ -602,7 +602,7 @@ generators:
   raise an exception inside the generator; the exception is raised by the
   ``yield`` expression where the generator's execution is paused.
 
-* :meth:`~generator.close` raises a :exc:`GeneratorExit` exception inside the
+* :meth:`~generator.close` sends a :exc:`GeneratorExit` exception to the
   generator to terminate the iteration.  On receiving this exception, the
   generator's code must either raise :exc:`GeneratorExit` or
   :exc:`StopIteration`; catching the exception and doing anything else is
@@ -1217,7 +1217,7 @@ flow inside a program.  The book uses Scheme for its examples, but many of the
 design approaches described in these chapters are applicable to functional-style
 Python code.
 
-https://www.defmacro.org/ramblings/fp.html: A general introduction to functional
+https://defmacro.org/2006/06/19/fp.html: A general introduction to functional
 programming that uses Java examples and has a lengthy historical introduction.
 
 https://en.wikipedia.org/wiki/Functional_programming: General Wikipedia entry

Doc/howto/isolating-extensions.rst

@@ -453,7 +453,7 @@ Avoiding ``PyObject_New``
 
 GC-tracked objects need to be allocated using GC-aware functions.
 
-If you use use :c:func:`PyObject_New` or :c:func:`PyObject_NewVar`:
+If you use :c:func:`PyObject_New` or :c:func:`PyObject_NewVar`:
 
 - Get and call type's :c:member:`~PyTypeObject.tp_alloc` slot, if possible.
   That is, replace ``TYPE *o = PyObject_New(TYPE, typeobj)`` with::
@@ -626,8 +626,7 @@ Open Issues
 
 Several issues around per-module state and heap types are still open.
 
-Discussions about improving the situation are best held on the `capi-sig
-mailing list <https://mail.python.org/mailman3/lists/capi-sig.python.org/>`__.
+Discussions about improving the situation are best held on the `discuss forum under c-api tag <https://discuss.python.org/c/core-dev/c-api/30>`__.
 
 
 Per-Class Scope

Doc/installing/index.rst

@@ -188,7 +188,7 @@ switch::
    Once the Development & Deployment part of PPUG is fleshed out, some of
    those sections should be linked from new questions here (most notably,
    we should have a question about avoiding depending on PyPI that links to
-   https://packaging.python.org/en/latest/mirrors/)
+   https://packaging.python.org/en/latest/guides/index-mirrors-and-caches/)
 
 
 Common installation issues

Doc/library/argparse.rst

@@ -839,23 +839,11 @@ how the command-line arguments should be handled. The supplied actions are:
     >>> parser.parse_args(['--version'])
     PROG 2.0
 
-Only actions that consume command-line arguments (e.g. ``'store'``,
-``'append'`` or ``'extend'``) can be used with positional arguments.
-
-.. class:: BooleanOptionalAction
-
-   You may also specify an arbitrary action by passing an :class:`Action` subclass or
-   other object that implements the same interface. The :class:`!BooleanOptionalAction`
-   is available in :mod:`!argparse` and adds support for boolean actions such as
-   ``--foo`` and ``--no-foo``::
-
-       >>> import argparse
-       >>> parser = argparse.ArgumentParser()
-       >>> parser.add_argument('--foo', action=argparse.BooleanOptionalAction)
-       >>> parser.parse_args(['--no-foo'])
-       Namespace(foo=False)
-
-   .. versionadded:: 3.9
+You may also specify an arbitrary action by passing an :class:`Action` subclass
+(e.g. :class:`BooleanOptionalAction`) or other object that implements the same
+interface. Only actions that consume command-line arguments (e.g. ``'store'``,
+``'append'``, ``'extend'``, or custom actions with non-zero ``nargs``) can be used
+with positional arguments.
 
 The recommended way to create a custom action is to extend :class:`Action`,
 overriding the :meth:`!__call__` method and optionally the :meth:`!__init__` and
@@ -955,7 +943,7 @@ See also :ref:`specifying-ambiguous-arguments`. The supported values are:
 
 .. index:: single: + (plus); in argparse module
 
-* ``'+'``. Just like ``'*'``, all command-line args present are gathered into a
+* ``'+'``. Just like ``'*'``, all command-line arguments present are gathered into a
   list.  Additionally, an error message will be generated if there wasn't at
   least one command-line argument present.  For example::
 
@@ -1429,6 +1417,21 @@ this API may be passed as the ``action`` parameter to
       and return a string which will be used when printing the usage of the program.
       If such method is not provided, a sensible default will be used.
 
+.. class:: BooleanOptionalAction
+
+   A subclass of :class:`Action` for handling boolean flags with positive
+   and negative options. Adding a single argument such as ``--foo`` automatically
+   creates both ``--foo`` and ``--no-foo`` options, storing ``True`` and ``False``
+   respectively::
+
+       >>> import argparse
+       >>> parser = argparse.ArgumentParser()
+       >>> parser.add_argument('--foo', action=argparse.BooleanOptionalAction)
+       >>> parser.parse_args(['--no-foo'])
+       Namespace(foo=False)
+
+   .. versionadded:: 3.9
+
 
 The parse_args() method
 -----------------------

Doc/library/codecs.rst

@@ -53,6 +53,14 @@ any codec:
    :exc:`UnicodeDecodeError`). Refer to :ref:`codec-base-classes` for more
    information on codec error handling.
 
+.. function:: charmap_build(string)
+
+   Return a mapping suitable for encoding with a custom single-byte encoding.
+   Given a :class:`str` *string* of up to 256 characters representing a
+   decoding table, returns either a compact internal mapping object
+   ``EncodingMap`` or a :class:`dictionary <dict>` mapping character ordinals
+   to byte values. Raises a :exc:`TypeError` on invalid input.
+
 The full details for each codec can also be looked up directly:
 
 .. function:: lookup(encoding, /)