[pull] main from python:main

Doc/extending/extending.rst

@@ -1106,7 +1106,7 @@ Finally it should be mentioned that Capsules offer additional functionality,
 which is especially useful for memory allocation and deallocation of the pointer
 stored in a Capsule. The details are described in the Python/C API Reference
 Manual in the section :ref:`capsules` and in the implementation of Capsules (files
-:file:`Include/pycapsule.h` and :file:`Objects/pycapsule.c` in the Python source
+:file:`Include/pycapsule.h` and :file:`Objects/capsule.c` in the Python source
 code distribution).
 
 .. rubric:: Footnotes

Doc/library/sqlite3.rst

@@ -1760,6 +1760,10 @@ Blob objects
 
    .. versionadded:: 3.11
 
+   .. versionchanged:: next
+      :class:`Blob` now supports negative-step slices
+      (e.g. ``blob[9:0:-2]``) for both reading and writing.
+
    A :class:`Blob` instance is a :term:`file-like object`
    that can read and write data in an SQLite :abbr:`BLOB (Binary Large OBject)`.
    Call :func:`len(blob) <len>` to get the size (number of bytes) of the blob.

Doc/library/sys.rst

@@ -1941,7 +1941,7 @@ always available. Unless explicitly noted otherwise, all variables are read-only
       The interpreter is about to execute a new line of code or re-execute the
       condition of a loop.  The local trace function is called; *arg* is
       ``None``; the return value specifies the new local trace function.  See
-      :file:`Objects/lnotab_notes.txt` for a detailed explanation of how this
+      :source:`InternalDocs/code_objects.md` for a detailed explanation of how this
       works.
       Per-line events may be disabled for a frame by setting
       :attr:`~frame.f_trace_lines` to :const:`False` on that

Doc/using/android.rst

@@ -37,7 +37,7 @@ much easier experience:
 * `Termux <https://termux.dev/en/>`__
 
 If you're sure you want to do all of this manually, read on. You can use the
-:source:`testbed app <Android/testbed>` as a guide; each step below contains a
+:source:`testbed app <Platforms/Android/testbed>` as a guide; each step below contains a
 link to the relevant file.
 
 * First, acquire a build of Python for Android:
@@ -47,10 +47,10 @@ link to the relevant file.
     mentioned below is at the top level of the package.
 
   * Or if you want to build it yourself, follow the instructions in
-    :source:`Android/README.md`. The ``prefix`` directory will be created under
+    :source:`Platforms/Android/README.md`. The ``prefix`` directory will be created under
     :samp:`cross-build/{HOST}`.
 
-* Add code to your :source:`build.gradle <Android/testbed/app/build.gradle.kts>`
+* Add code to your :source:`build.gradle <Platforms/Android/testbed/app/build.gradle.kts>`
   file to copy the following items into your project. All except your own Python
   code can be copied from ``prefix/lib``:
 
@@ -65,10 +65,10 @@ link to the relevant file.
     * ``python*.*/site-packages`` (your own Python code)
 
 * Add code to your app to :source:`extract the assets to the filesystem
-  <Android/testbed/app/src/main/java/org/python/testbed/MainActivity.kt>`.
+  <Platforms/Android/testbed/app/src/main/java/org/python/testbed/MainActivity.kt>`.
 
 * Add code to your app to :source:`start Python in embedded mode
-  <Android/testbed/app/src/main/c/main_activity.c>`. This will need to be C code
+  <Platforms/Android/testbed/app/src/main/c/main_activity.c>`. This will need to be C code
   called via JNI.
 
 Building a Python package for Android

Doc/using/configure.rst

@@ -71,7 +71,8 @@ Dependencies to build optional modules are:
    * - Dependency
      - Minimum version
      - Python module
-   * - `libbz2 <https://sourceware.org/bzip2/>`_
+   * - `libbz2 <https://sourceware.org/bzip2/>`_ or
+       `libbzip2-rs <https://github.com/trifectatechfoundation/libbzip2-rs>`_ [1]_
      -
      - :mod:`bz2`
    * - `libffi <https://sourceware.org/libffi/>`_
@@ -82,47 +83,54 @@ Dependencies to build optional modules are:
      - :mod:`lzma`
    * - `libmpdec <https://www.bytereef.org/mpdecimal/doc/libmpdec/>`_
      - 2.5.0
-     - :mod:`decimal` [1]_
+     - :mod:`decimal` [2]_
    * - `libreadline <https://tiswww.case.edu/php/chet/readline/rltop.html>`_ or
-       `libedit <https://www.thrysoee.dk/editline/>`_ [2]_
+       `libedit <https://www.thrysoee.dk/editline/>`_ [3]_
      -
      - :mod:`readline`
    * - `libuuid <https://linux.die.net/man/3/libuuid>`_
      -
-     - ``_uuid`` [3]_
-   * - `ncurses <https://gnu.org/software/ncurses/ncurses.html>`_ [4]_
+     - ``_uuid`` [4]_
+   * - `ncurses <https://gnu.org/software/ncurses/ncurses.html>`_ [5]_
      -
      - :mod:`curses`
    * - `OpenSSL <https://openssl-library.org/>`_
      - | 3.0.18 recommended
        | (1.1.1 minimum)
-     - :mod:`ssl`, :mod:`hashlib` [5]_
+     - :mod:`ssl`, :mod:`hashlib` [6]_
    * - `SQLite <https://sqlite.org/>`_
      - 3.15.2
      - :mod:`sqlite3`
    * - `Tcl/Tk <https://www.tcl-lang.org/>`_
      - 8.5.12
      - :mod:`tkinter`, :ref:`IDLE <idle>`, :mod:`turtle`
-   * - `zlib <https://www.zlib.net>`_
-     - 1.2.2.1
+   * - `zlib <https://www.zlib.net>`_,
+       `zlib-ng <https://github.com/zlib-ng/zlib-ng>`_, or
+       `zlib-rs <https://github.com/trifectatechfoundation/zlib-rs/>`_ [7]_
+     - | 1.2.2.1 (zlib, zlib-ng)
+       | 0.6.0 (zlib-rs)
      - :mod:`zlib`, :mod:`gzip`, :mod:`ensurepip`
    * - `zstd <https://facebook.github.io/zstd/>`_
      - 1.4.5
      - :mod:`compression.zstd`
 
-.. [1] If *libmpdec* is not available, the :mod:`decimal` module will use
+.. [1] See :option:`--with-bzip2` for choosing the backend for the
+   :mod:`bz2` module.
+.. [2] If *libmpdec* is not available, the :mod:`decimal` module will use
    a pure-Python implementation.
-.. [2] See :option:`--with-readline` for choosing the backend for the
+.. [3] See :option:`--with-readline` for choosing the backend for the
    :mod:`readline` module.
-.. [3] The :mod:`uuid` module uses ``_uuid`` to generate "safe" UUIDs.
+.. [4] The :mod:`uuid` module uses ``_uuid`` to generate "safe" UUIDs.
    See the module documentation for details.
-.. [4] The :mod:`curses` module requires the ``libncurses`` or ``libncursesw``
+.. [5] The :mod:`curses` module requires the ``libncurses`` or ``libncursesw``
    library.
    The :mod:`curses.panel` module additionally requires the ``libpanel`` or
    ``libpanelw`` library.
-.. [5] If OpenSSL is not available, the :mod:`hashlib` module will use
+.. [6] If OpenSSL is not available, the :mod:`hashlib` module will use
    bundled implementations of several hash functions.
    See :option:`--with-builtin-hashlib-hashes` for *forcing* usage of OpenSSL.
+.. [7] See :option:`--with-zlib` for choosing the backend for the
+   :mod:`zlib` module.
 
 Note that the table does not include all optional modules; in particular,
 platform-specific modules like :mod:`winreg` are not listed here.
@@ -1113,6 +1121,54 @@ Libraries options
 
    .. versionadded:: next
 
+.. option:: --with-zlib=zlib|zlib-ng|zlib-rs
+
+   Designate the zlib implementation to build the :mod:`zlib` module against.
+
+   * ``zlib``: Use the system zlib library, found via ``pkg-config`` or as
+     ``libz``.
+   * ``zlib-ng``: Same detection as ``zlib``, but fail unless the detected
+     library is `zlib-ng <https://github.com/zlib-ng/zlib-ng>`__.
+   * ``zlib-rs``: Use `zlib-rs
+     <https://github.com/trifectatechfoundation/zlib-rs/>`__, found via
+     ``pkg-config`` as ``libz_rs``.
+
+   Without this option (or with ``--with-zlib=auto``, the default), the system
+   zlib is used when found; configure only fails when an implementation was
+   requested explicitly but cannot be found.
+
+   .. versionadded:: next
+
+.. option:: --without-zlib
+
+   Don't build the :mod:`zlib` module, even when a zlib library is present
+   (built by default when one is found).
+
+   .. versionadded:: next
+
+.. option:: --with-bzip2=bzip2|bzip2-rs
+
+   Designate the bzip2 implementation to build the :mod:`bz2` module against.
+
+   * ``bzip2``: Use the system bzip2 library, found via ``pkg-config`` or as
+     ``libbz2``.
+   * ``bzip2-rs``: Use `libbzip2-rs
+     <https://github.com/trifectatechfoundation/libbzip2-rs>`__, found via
+     ``pkg-config`` as ``libbz2_rs``.
+
+   Without this option (or with ``--with-bzip2=auto``, the default), the
+   system bzip2 is used when found; configure only fails when an
+   implementation was requested explicitly but cannot be found.
+
+   .. versionadded:: next
+
+.. option:: --without-bzip2
+
+   Don't build the :mod:`bz2` module, even when a bzip2 library is present
+   (built by default when one is found).
+
+   .. versionadded:: next
+
 .. option:: --with-libm=STRING
 
    Override ``libm`` math library to *STRING* (default is system-dependent).
@@ -1275,7 +1331,7 @@ See :source:`Mac/README.rst`.
 iOS Options
 -----------
 
-See :source:`iOS/README.rst`.
+See :source:`Platforms/Apple/iOS/README.md`.
 
 .. option:: --enable-framework=INSTALLDIR
 

Doc/using/ios.rst

@@ -170,7 +170,7 @@ helpful.
 To add Python to an iOS Xcode project:
 
 1. Build or obtain a Python ``XCFramework``. See the instructions in
-   :source:`Apple/iOS/README.md` (in the CPython source distribution) for details on
+   :source:`Platforms/Apple/iOS/README.md` (in the CPython source distribution) for details on
    how to build a Python ``XCFramework``. At a minimum, you will need a build
    that supports ``arm64-apple-ios``, plus one of either
    ``arm64-apple-ios-simulator`` or ``x86_64-apple-ios-simulator``.
@@ -266,13 +266,13 @@ modules in your app, some additional steps will be required:
 Testing a Python package
 ------------------------
 
-The CPython source tree contains :source:`a testbed project <Apple/iOS/testbed>` that
+The CPython source tree contains :source:`a testbed project <Platforms/Apple/testbed>` that
 is used to run the CPython test suite on the iOS simulator. This testbed can also
 be used as a testbed project for running your Python library's test suite on iOS.
 
-After building or obtaining an iOS XCFramework (see :source:`Apple/iOS/README.md`
+After building or obtaining an iOS XCFramework (see :source:`Platforms/Apple/iOS/README.md`
 for details), create a clone of the Python iOS testbed project. If you used the
-``Apple`` build script to build the XCframework, you can run:
+``Platforms/Apple`` build script to build the XCframework, you can run:
 
 .. code-block:: bash
 
@@ -282,7 +282,7 @@ Or, if you've sourced your own XCframework, by running:
 
 .. code-block:: bash
 
-    $ python Apple/testbed clone --platform iOS --framework <path/to/Python.xcframework> --app <path/to/module1> --app <path/to/module2> app-testbed
+    $ python Platforms/Apple/testbed clone --platform iOS --framework <path/to/Python.xcframework> --app <path/to/module1> --app <path/to/module2> app-testbed
 
 Any folders specified with the ``--app`` flag will be copied into the cloned
 testbed project. The resulting testbed will be created in the ``app-testbed``

Doc/whatsnew/3.16.rst

@@ -466,6 +466,13 @@ shlex
   a string, even if it is already safe for a shell without being quoted.
   (Contributed by Jay Berry in :gh:`148846`.)
 
+sqlite3
+-------
+
+* :class:`sqlite3.Blob` now supports negative-step slices for reading and
+  writing (e.g. ``blob[9:0:-2]``).  Previously, such slices would raise
+  :exc:`SystemError` or :exc:`ValueError`.
+  (Contributed by Jiseok CHOI in :gh:`150449`.)
 
 symtable
 --------
@@ -870,6 +877,18 @@ Build changes
 
   (Contributed by Serhiy Storchaka in :gh:`136687`.)
 
+* Add the :option:`--with-zlib` and :option:`--with-bzip2` :program:`configure`
+  options to select the zlib and bzip2 implementations used to build the
+  :mod:`zlib` and :mod:`bz2` modules. In addition to the default system
+  libraries, they can build against `zlib-rs
+  <https://github.com/trifectatechfoundation/zlib-rs/>`__ and `libbzip2-rs
+  <https://github.com/trifectatechfoundation/libbzip2-rs>`__, and
+  ``--with-zlib=zlib-ng`` verifies that the detected zlib is `zlib-ng
+  <https://github.com/zlib-ng/zlib-ng>`__. :option:`--without-zlib` and
+  :option:`--without-bzip2` exclude the modules from the build.
+
+  (Contributed by Stan Ulbrych in :gh:`139314`.)
+
 
 C API changes
 =============

InternalDocs/code_objects.md

@@ -48,7 +48,7 @@ Note that traceback objects don't store all this information -- they store the s
 number, for backward compatibility, and the "last instruction" value.
 The rest can be computed from the last instruction (`tb_lasti`) with the help of the
 locations table. For Python code, there is a convenience method
-(`codeobject.co_positions`)[https://docs.python.org/dev/reference/datamodel.html#codeobject.co_positions]
+[`codeobject.co_positions`](https://docs.python.org/dev/reference/datamodel.html#codeobject.co_positions)
 which returns an iterator of `({line}, {endline}, {column}, {endcolumn})` tuples,
 one per instruction.
 There is also `co_lines()` which returns an iterator of `({start}, {end}, {line})` tuples,

InternalDocs/jit.md

@@ -140,7 +140,7 @@ template file [`Tools/jit/template.c`](../Tools/jit/template.c).
 Each of the `.c` files is compiled by LLVM, to produce an object file
 that contains a function that executes the opcode. These compiled
 functions are used to generate the file
-[`jit_stencils.h`](../jit_stencils.h), which contains the functions
+`jit_stencils.h`, which contains the functions
 that the JIT can use to emit code for each of the bytecodes.
 
 For Python maintainers this means that changes to the bytecodes and

Lib/imaplib.py

@@ -168,6 +168,8 @@ def _seq_range(item):
 
 
 def _format_sequence_set(arg):
+    if isinstance(arg, (bytes, bytearray)):
+        return str(arg, 'ascii')
     if isinstance(arg, (int, str)):
         return str(arg)
     # A sequence of message numbers and ranges.

Lib/test/test_imaplib.py

@@ -292,6 +292,8 @@ def test_sequence_set(self):
         # A scalar is passed through as a string.
         self.assertEqual(m._sequence_set(5), '5')
         self.assertEqual(m._sequence_set('1:3,7'), '1:3,7')
+        self.assertEqual(m._sequence_set(b'1:3,7'), '1:3,7')
+        self.assertEqual(m._sequence_set(bytearray(b'1:3,7')), '1:3,7')
         # A sequence of numbers and ranges is formatted as a sequence set.
         self.assertEqual(m._sequence_set([1, 2, 5]), '1,2,5')
         self.assertEqual(m._sequence_set([1, (3, 5), (8, '*')]), '1,3:5,8:*')
@@ -333,6 +335,7 @@ def test_substitute(self):
                          r'(\Seen \Answered)')
         # '?s' formats a message sequence set.
         self.assertEqual(sub('?s', [[1, (3, 5), (8, '*')]]), '1,3:5,8:*')
+        self.assertEqual(sub('?s', [b'1:3,7']), '1:3,7')
         # '??' is a literal '?'.
         self.assertEqual(sub('a?? b', []), 'a? b')
 
@@ -1574,6 +1577,16 @@ def cmd_FETCH(self, tag, args):
         ])
         self.assertEqual(server.args, ['2:4', '(FLAGS)'])
 
+        # A preformatted message set may be passed as bytes.
+        typ, data = client.fetch(b'2:4', '(FLAGS)')
+        self.assertEqual(typ, 'OK')
+        self.assertEqual(data, [
+            br'2 (FLAGS (\Seen))',
+            br'3 (FLAGS (\Seen))',
+            br'4 (FLAGS (\Seen))',
+        ])
+        self.assertEqual(server.args, ['2:4', '(FLAGS)'])
+
         # A comma-separated set with an open range up to '*'.
         typ, data = client.fetch('1,3:*', '(FLAGS)')
         self.assertEqual(typ, 'OK')

Lib/test/test_sqlite3/test_dbapi.py

@@ -1390,6 +1390,19 @@ def test_blob_get_slice_negative_index(self):
     def test_blob_get_slice_with_skip(self):
         self.assertEqual(self.blob[0:10:2], b"ti lb")
 
+    def test_blob_get_slice_with_negative_step(self):
+        # gh-150449: negative-step slices must not crash
+        self.assertEqual(self.blob[9:0:-2], self.data[9:0:-2])
+        self.assertEqual(self.blob[9::-2], self.data[9::-2])
+        self.assertEqual(self.blob[::-1], self.data[::-1])
+        # When start <= stop with a negative step the slice is empty; this
+        # must return b"" rather than crashing or raising an exception.
+        self.assertEqual(self.blob[3:8:-1], self.data[3:8:-1])   # b""
+        self.assertEqual(self.blob[5:5:-1], self.data[5:5:-1])   # b""
+        # Extreme step values: cur += (size_t)step must not overflow.
+        self.assertEqual(self.blob[5::sys.maxsize], self.data[5::sys.maxsize])
+        self.assertEqual(self.blob[::-sys.maxsize - 1], self.data[::-sys.maxsize - 1])
+
     def test_blob_set_slice(self):
         self.blob[0:5] = b"12345"
         expected = b"12345" + self.data[5:]
@@ -1430,6 +1443,43 @@ def test_blob_set_slice_with_skip(self):
         expected = b"1h2s3b4o5 " + self.data[10:]
         self.assertEqual(actual, expected)
 
+    def test_blob_set_slice_with_negative_step(self):
+        # gh-150449: negative-step slice assignment must not crash
+        expected = bytearray(self.data)
+        expected[9:0:-2] = b"12345"
+        self.blob[9:0:-2] = b"12345"
+        actual = self.cx.execute("select b from test").fetchone()[0]
+        self.assertEqual(actual, bytes(expected))
+
+        # Also verify a slice that includes index 0
+        expected2 = bytearray(self.data)
+        expected2[9::-2] = b"12345"
+        self.blob[9::-2] = b"12345"
+        actual2 = self.cx.execute("select b from test").fetchone()[0]
+        self.assertEqual(actual2, bytes(expected2))
+
+        # When start <= stop with a negative step the slice is empty;
+        # assigning b"" to it must be a no-op (blob contents unchanged).
+        state_before = bytes(self.blob[:])
+        self.blob[3:8:-1] = b""
+        self.assertEqual(bytes(self.blob[:]), state_before)
+
+    def test_blob_set_slice_with_extreme_positive_step(self):
+        expected = bytearray(self.data)
+        expected[5::sys.maxsize] = b"\xab"
+        self.blob[5::sys.maxsize] = b"\xab"
+        actual = self.cx.execute("select b from test").fetchone()[0]
+        self.assertEqual(actual, bytes(expected))
+        self.assertEqual(actual[5], 0xab)
+
+    def test_blob_set_slice_with_extreme_negative_step(self):
+        expected = bytearray(self.data)
+        expected[::-sys.maxsize - 1] = b"\xcd"
+        self.blob[::-sys.maxsize - 1] = b"\xcd"
+        actual = self.cx.execute("select b from test").fetchone()[0]
+        self.assertEqual(actual, bytes(expected))
+        self.assertEqual(actual[-1], 0xcd)
+
     def test_blob_mapping_invalid_index_type(self):
         msg = "indices must be integers"
         with self.assertRaisesRegex(TypeError, msg):