Merge branch 'python:main' into main

.github/workflows/reusable-docs.yml

@@ -99,7 +99,7 @@ jobs:
   # Run "doctest" on HEAD as new syntax doesn't exist in the latest stable release
   doctest:
     name: 'Doctest'
-    runs-on: ubuntu-22.04
+    runs-on: ubuntu-24.04
     timeout-minutes: 60
     steps:
     - uses: actions/checkout@v4

Doc/c-api/object.rst

@@ -493,6 +493,13 @@ Object Protocol
    on failure.  This is equivalent to the Python statement ``del o[key]``.
 
 
+.. c:function:: int PyObject_DelItemString(PyObject *o, const char *key)
+
+   This is the same as :c:func:`PyObject_DelItem`, but *key* is
+   specified as a :c:expr:`const char*` UTF-8 encoded bytes string,
+   rather than a :c:expr:`PyObject*`.
+
+
 .. c:function:: PyObject* PyObject_Dir(PyObject *o)
 
    This is equivalent to the Python expression ``dir(o)``, returning a (possibly

Doc/library/json.rst

@@ -151,69 +151,94 @@ Basic Usage
                    sort_keys=False, **kw)
 
    Serialize *obj* as a JSON formatted stream to *fp* (a ``.write()``-supporting
-   :term:`file-like object`) using this :ref:`conversion table
+   :term:`file-like object`) using this :ref:`Python-to-JSON conversion table
    <py-to-json-table>`.
 
-   If *skipkeys* is true (default: ``False``), then dict keys that are not
-   of a basic type (:class:`str`, :class:`int`, :class:`float`, :class:`bool`,
-   ``None``) will be skipped instead of raising a :exc:`TypeError`.
-
-   The :mod:`json` module always produces :class:`str` objects, not
-   :class:`bytes` objects. Therefore, ``fp.write()`` must support :class:`str`
-   input.
-
-   If *ensure_ascii* is true (the default), the output is guaranteed to
-   have all incoming non-ASCII characters escaped.  If *ensure_ascii* is
-   false, these characters will be output as-is.
-
-   If *check_circular* is false (default: ``True``), then the circular
-   reference check for container types will be skipped and a circular reference
-   will result in a :exc:`RecursionError` (or worse).
+   .. note::
 
-   If *allow_nan* is false (default: ``True``), then it will be a
-   :exc:`ValueError` to serialize out of range :class:`float` values (``nan``,
-   ``inf``, ``-inf``) in strict compliance of the JSON specification.
-   If *allow_nan* is true, their JavaScript equivalents (``NaN``,
-   ``Infinity``, ``-Infinity``) will be used.
+      Unlike :mod:`pickle` and :mod:`marshal`, JSON is not a framed protocol,
+      so trying to serialize multiple objects with repeated calls to
+      :func:`dump` using the same *fp* will result in an invalid JSON file.
 
-   If *indent* is a non-negative integer or string, then JSON array elements and
-   object members will be pretty-printed with that indent level.  An indent level
-   of 0, negative, or ``""`` will only insert newlines.  ``None`` (the default)
-   selects the most compact representation. Using a positive integer indent
-   indents that many spaces per level.  If *indent* is a string (such as ``"\t"``),
-   that string is used to indent each level.
+   :param object obj:
+      The Python object to be serialized.
+
+   :param fp:
+      The file-like object *obj* will be serialized to.
+      The :mod:`!json` module always produces :class:`str` objects,
+      not :class:`bytes` objects,
+      therefore ``fp.write()`` must support :class:`str` input.
+   :type fp: :term:`file-like object`
+
+   :param bool skipkeys:
+      If ``True``, keys that are not of a basic type
+      (:class:`str`, :class:`int`, :class:`float`, :class:`bool`, ``None``)
+      will be skipped instead of raising a :exc:`TypeError`.
+      Default ``False``.
+
+   :param bool ensure_ascii:
+      If ``True`` (the default), the output is guaranteed to
+      have all incoming non-ASCII characters escaped.
+      If ``False``, these characters will be outputted as-is.
+
+   :param bool check_circular:
+      If ``False``, the circular reference check for container types is skipped
+      and a circular reference will result in a :exc:`RecursionError` (or worse).
+      Default ``True``.
+
+   :param bool allow_nan:
+      If ``False``, serialization of out-of-range :class:`float` values
+      (``nan``, ``inf``, ``-inf``) will result in a :exc:`ValueError`,
+      in strict compliance with the JSON specification.
+      If ``True`` (the default), their JavaScript equivalents
+      (``NaN``, ``Infinity``, ``-Infinity``) are used.
+
+   :param cls:
+      If set, a custom JSON encoder with the
+      :meth:`~JSONEncoder.default` method overridden,
+      for serializing into custom datatypes.
+      If ``None`` (the default), :class:`!JSONEncoder` is used.
+   :type cls: a :class:`JSONEncoder` subclass
+
+   :param indent:
+      If a positive integer or string, JSON array elements and
+      object members will be pretty-printed with that indent level.
+      A positive integer indents that many spaces per level;
+      a string (such as ``"\t"``) is used to indent each level.
+      If zero, negative, or ``""`` (the empty string),
+      only newlines are inserted.
+      If ``None`` (the default), the most compact representation is used.
+   :type indent: int | str | None
+
+   :param separators:
+      A two-tuple: ``(item_separator, key_separator)``.
+      If ``None`` (the default), *separators* defaults to
+      ``(', ', ': ')`` if *indent* is ``None``,
+      and ``(',', ': ')`` otherwise.
+      For the most compact JSON,
+      specify ``(',', ':')`` to eliminate whitespace.
+   :type separators: tuple | None
+
+   :param default:
+      A function that is called for objects that can't otherwise be serialized.
+      It should return a JSON encodable version of the object
+      or raise a :exc:`TypeError`.
+      If ``None`` (the default), :exc:`!TypeError` is raised.
+   :type default: :term:`callable` | None
+
+   :param bool sort_keys:
+      If ``True``, dictionaries will be outputted sorted by key.
+      Default ``False``.
 
    .. versionchanged:: 3.2
       Allow strings for *indent* in addition to integers.
 
-   If specified, *separators* should be an ``(item_separator, key_separator)``
-   tuple.  The default is ``(', ', ': ')`` if *indent* is ``None`` and
-   ``(',', ': ')`` otherwise.  To get the most compact JSON representation,
-   you should specify ``(',', ':')`` to eliminate whitespace.
-
    .. versionchanged:: 3.4
       Use ``(',', ': ')`` as default if *indent* is not ``None``.
 
-   If specified, *default* should be a function that gets called for objects that
-   can't otherwise be serialized.  It should return a JSON encodable version of
-   the object or raise a :exc:`TypeError`.  If not specified, :exc:`TypeError`
-   is raised.
-
-   If *sort_keys* is true (default: ``False``), then the output of
-   dictionaries will be sorted by key.
-
-   To use a custom :class:`JSONEncoder` subclass (e.g. one that overrides the
-   :meth:`~JSONEncoder.default` method to serialize additional types), specify it with the
-   *cls* kwarg; otherwise :class:`JSONEncoder` is used.
-
    .. versionchanged:: 3.6
       All optional parameters are now :ref:`keyword-only <keyword-only_parameter>`.
 
-   .. note::
-
-      Unlike :mod:`pickle` and :mod:`marshal`, JSON is not a framed protocol,
-      so trying to serialize multiple objects with repeated calls to
-      :func:`dump` using the same *fp* will result in an invalid JSON file.
 
 .. function:: dumps(obj, *, skipkeys=False, ensure_ascii=True, \
                     check_circular=True, allow_nan=True, cls=None, \

Doc/library/math.rst

@@ -248,7 +248,8 @@ Floating point arithmetic
 
 .. function:: fmod(x, y)
 
-   Return ``fmod(x, y)``, as defined by the platform C library. Note that the
+   Return the floating-point remainder of ``x / y``,
+   as defined by the platform C library function ``fmod(x, y)``. Note that the
    Python expression ``x % y`` may not return the same result.  The intent of the C
    standard is that ``fmod(x, y)`` be exactly (mathematically; to infinite
    precision) equal to ``x - n*y`` for some integer *n* such that the result has

Doc/using/configure.rst

@@ -29,7 +29,7 @@ Features and minimum versions required to build CPython:
 
 * Tcl/Tk 8.5.12 for the :mod:`tkinter` module.
 
-* Autoconf 2.71 and aclocal 1.16.5 are required to regenerate the
+* Autoconf 2.72 and aclocal 1.16.5 are required to regenerate the
   :file:`configure` script.
 
 .. versionchanged:: 3.1
@@ -58,6 +58,9 @@ Features and minimum versions required to build CPython:
 .. versionchanged:: 3.13
    Autoconf 2.71, aclocal 1.16.5 and SQLite 3.15.2 are now required.
 
+.. versionchanged:: next
+   Autoconf 2.72 is now required.
+
 See also :pep:`7` "Style Guide for C Code" and :pep:`11` "CPython platform
 support".
 

Include/cpython/unicodeobject.h

@@ -109,7 +109,7 @@ typedef struct {
                3: Interned, Immortal, and Static
            This categorization allows the runtime to determine the right
            cleanup mechanism at runtime shutdown. */
-        unsigned int interned:2;
+        uint16_t interned;
         /* Character size:
 
            - PyUnicode_1BYTE_KIND (1):
@@ -132,21 +132,23 @@ typedef struct {
              * all characters are in the range U+0000-U+10FFFF
              * at least one character is in the range U+10000-U+10FFFF
          */
-        unsigned int kind:3;
+        unsigned short kind:3;
         /* Compact is with respect to the allocation scheme. Compact unicode
            objects only require one memory block while non-compact objects use
            one block for the PyUnicodeObject struct and another for its data
            buffer. */
-        unsigned int compact:1;
+        unsigned short compact:1;
         /* The string only contains characters in the range U+0000-U+007F (ASCII)
            and the kind is PyUnicode_1BYTE_KIND. If ascii is set and compact is
            set, use the PyASCIIObject structure. */
-        unsigned int ascii:1;
+        unsigned short ascii:1;
         /* The object is statically allocated. */
-        unsigned int statically_allocated:1;
+        unsigned short statically_allocated:1;
         /* Padding to ensure that PyUnicode_DATA() is always aligned to
-           4 bytes (see issue #19537 on m68k). */
-        unsigned int :24;
+           4 bytes (see issue #19537 on m68k) and we use unsigned short to avoid
+           the extra four bytes on 32-bit Windows. This is restricted features
+           for specific compilers including GCC, MSVC, Clang and IBM's XL compiler. */
+        unsigned short :10;
     } state;
 } PyASCIIObject;
 
@@ -195,7 +197,11 @@ typedef struct {
 
 /* Use only if you know it's a string */
 static inline unsigned int PyUnicode_CHECK_INTERNED(PyObject *op) {
+#ifdef Py_GIL_DISABLED
+    return _Py_atomic_load_uint16_relaxed(&_PyASCIIObject_CAST(op)->state.interned);
+#else
     return _PyASCIIObject_CAST(op)->state.interned;
+#endif
 }
 #define PyUnicode_CHECK_INTERNED(op) PyUnicode_CHECK_INTERNED(_PyObject_CAST(op))
 

Lib/_pydatetime.py

@@ -2392,7 +2392,6 @@ def __reduce__(self):
 
 def _isoweek1monday(year):
     # Helper to calculate the day number of the Monday starting week 1
-    # XXX This could be done more efficiently
     THURSDAY = 3
     firstday = _ymd2ord(year, 1, 1)
     firstweekday = (firstday + 6) % 7  # See weekday() above

Lib/test/test_pathlib/test_pathlib.py

@@ -3029,6 +3029,42 @@ def setUp(self):
         if name in _tests_needing_symlinks and not self.can_symlink:
             self.skipTest('requires symlinks')
         super().setUp()
+
+    def createTestHierarchy(self):
+        # Build:
+        #     TESTFN/
+        #       TEST1/              a file kid and two directory kids
+        #         tmp1
+        #         SUB1/             a file kid and a directory kid
+        #           tmp2
+        #           SUB11/          no kids
+        #         SUB2/             a file kid and a dirsymlink kid
+        #           tmp3
+        #           link/           a symlink to TEST2
+        #           broken_link
+        #           broken_link2
+        #       TEST2/
+        #         tmp4              a lone file
+        t2_path = self.cls(self.base, "TEST2")
+        os.makedirs(self.sub11_path)
+        os.makedirs(self.sub2_path)
+        os.makedirs(t2_path)
+
+        tmp1_path = self.walk_path / "tmp1"
+        tmp2_path = self.sub1_path / "tmp2"
+        tmp3_path = self.sub2_path / "tmp3"
+        tmp4_path = self.cls(self.base, "TEST2", "tmp4")
+        for path in tmp1_path, tmp2_path, tmp3_path, tmp4_path:
+            with open(path, "w", encoding='utf-8') as f:
+                f.write(f"I'm {path} and proud of it.  Blame test_pathlib.\n")
+
+        if self.can_symlink:
+            broken_link_path = self.sub2_path / "broken_link"
+            broken_link2_path = self.sub2_path / "broken_link2"
+            os.symlink(t2_path, self.link_path, target_is_directory=True)
+            os.symlink('broken', broken_link_path)
+            os.symlink(os.path.join('tmp3', 'broken'), broken_link2_path)
+            self.sub2_tree = (self.sub2_path, [], ["broken_link", "broken_link2", "link", "tmp3"])
         sub21_path= self.sub2_path / "SUB21"
         tmp5_path = sub21_path / "tmp3"
         broken_link3_path = self.sub2_path / "broken_link3"
@@ -3052,7 +3088,7 @@ def setUp(self):
     def tearDown(self):
         if 'SUB21' in self.sub2_tree[1]:
             os.chmod(self.sub2_path / "SUB21", stat.S_IRWXU)
-        super().tearDown()
+        os_helper.rmtree(self.base)
 
     def test_walk_bad_dir(self):
         errors = []

Lib/test/test_pathlib/test_pathlib_abc.py

@@ -1580,52 +1580,35 @@ class DummyPathWalkTest(unittest.TestCase):
     can_symlink = False
 
     def setUp(self):
-        # Build:
-        #     TESTFN/
-        #       TEST1/              a file kid and two directory kids
-        #         tmp1
-        #         SUB1/             a file kid and a directory kid
-        #           tmp2
-        #           SUB11/          no kids
-        #         SUB2/             a file kid and a dirsymlink kid
-        #           tmp3
-        #           link/           a symlink to TEST2
-        #           broken_link
-        #           broken_link2
-        #       TEST2/
-        #         tmp4              a lone file
         self.walk_path = self.cls(self.base, "TEST1")
         self.sub1_path = self.walk_path / "SUB1"
         self.sub11_path = self.sub1_path / "SUB11"
         self.sub2_path = self.walk_path / "SUB2"
-        tmp1_path = self.walk_path / "tmp1"
-        tmp2_path = self.sub1_path / "tmp2"
-        tmp3_path = self.sub2_path / "tmp3"
         self.link_path = self.sub2_path / "link"
-        t2_path = self.cls(self.base, "TEST2")
-        tmp4_path = self.cls(self.base, "TEST2", "tmp4")
-        broken_link_path = self.sub2_path / "broken_link"
-        broken_link2_path = self.sub2_path / "broken_link2"
-
-        self.sub11_path.mkdir(parents=True)
-        self.sub2_path.mkdir(parents=True)
-        t2_path.mkdir(parents=True)
-
-        for path in tmp1_path, tmp2_path, tmp3_path, tmp4_path:
-            with path.open("w", encoding='utf-8') as f:
-                f.write(f"I'm {path} and proud of it.  Blame test_pathlib.\n")
+        self.sub2_tree = (self.sub2_path, [], ["tmp3"])
+        self.createTestHierarchy()
 
-        if self.can_symlink:
-            self.link_path.symlink_to(t2_path, target_is_directory=True)
-            broken_link_path.symlink_to('broken')
-            broken_link2_path.symlink_to(self.cls('tmp3', 'broken'))
-            self.sub2_tree = (self.sub2_path, [], ["broken_link", "broken_link2", "link", "tmp3"])
-        else:
-            self.sub2_tree = (self.sub2_path, [], ["tmp3"])
+    def createTestHierarchy(self):
+        cls = self.cls
+        cls._files = {
+            f'{self.base}/TEST1/tmp1': b'this is tmp1\n',
+            f'{self.base}/TEST1/SUB1/tmp2': b'this is tmp2\n',
+            f'{self.base}/TEST1/SUB2/tmp3': b'this is tmp3\n',
+            f'{self.base}/TEST2/tmp4': b'this is tmp4\n',
+        }
+        cls._directories = {
+            f'{self.base}': {'TEST1', 'TEST2'},
+            f'{self.base}/TEST1': {'SUB1', 'SUB2', 'tmp1'},
+            f'{self.base}/TEST1/SUB1': {'SUB11', 'tmp2'},
+            f'{self.base}/TEST1/SUB1/SUB11': set(),
+            f'{self.base}/TEST1/SUB2': {'tmp3'},
+            f'{self.base}/TEST2': {'tmp4'},
+        }
 
     def tearDown(self):
-        base = self.cls(self.base)
-        base._delete()
+        cls = self.cls
+        cls._files.clear()
+        cls._directories.clear()
 
     def test_walk_topdown(self):
         walker = self.walk_path.walk()

Misc/NEWS.d/next/Core_and_Builtins/2024-12-24-01-40-12.gh-issue-128137.gsTwr_.rst

@@ -0,0 +1,2 @@
+Update :c:type:`PyASCIIObject` layout to handle interned field with the
+atomic operation. Patch by Donghee Na.