Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
11f1822
PYTHON-6077 Officially support Python 3.15
blink1073 Sep 4, 2026
82afc1a
PYTHON-6077 Always use UV_PYTHON and prefer toolchain pythons
blink1073 Sep 4, 2026
4098a2e
Merge remote-tracking branch 'upstream/main' into worktree-PYTHON-607…
blink1073 Sep 5, 2026
78b8a2e
PYTHON-6077 Drop the python-build-standalone fallback for 3.15
blink1073 Sep 5, 2026
620507f
Revert "PYTHON-6077 Drop the python-build-standalone fallback for 3.15"
blink1073 Sep 5, 2026
f6be3d0
PYTHON-6077 Handle remaining 3.15 CI failures
blink1073 Sep 5, 2026
4841077
PYTHON-6077 Generalize python-build-standalone fetch for any missing …
blink1073 Sep 8, 2026
2a2c0c6
PYTHON-6077 Clarify python download conditions in setup-dev-env
blink1073 Sep 8, 2026
a70b3a5
Handle x86_64 architecture for Windows and others
blink1073 Sep 8, 2026
5a34041
Refactor Python interpreter setup in setup-dev-env.sh
blink1073 Sep 8, 2026
f3acdf1
PYTHON-6077 Fix missing fi in setup-dev-env.sh
blink1073 Sep 8, 2026
133c3bb
PYTHON-6077 Accept unpadded/whitespace-padded keytab on 3.15
blink1073 Sep 8, 2026
1f56904
PYTHON-6077 Strip internal whitespace from keytab base64
blink1073 Sep 8, 2026
473ed18
PYTHON-6077 Decode keytab with lenient padding on 3.15
blink1073 Sep 8, 2026
6501c2f
Merge remote-tracking branch 'upstream/main' into worktree-PYTHON-607…
blink1073 Sep 8, 2026
778de06
PYTHON-6077 Make test_list_collections robust to empty test db
blink1073 Sep 8, 2026
27bcd1e
PYTHON-6077 Clone drivers-tools module regardless of env.sh
blink1073 Sep 8, 2026
269e79b
PYTHON-6077 Use PATH-based Python resolution with bare UV_PYTHON
blink1073 Sep 8, 2026
95e0e90
PYTHON-6077 Scope UV_PYTHON_PREFERENCE=system to non-Windows
blink1073 Sep 8, 2026
ea36a45
PYTHON-6077 Use path-based UV_PYTHON on Windows
blink1073 Sep 9, 2026
461b5b1
PYTHON-6077 Skip fetch fallback on re-entry by marking Python resolved
blink1073 Sep 9, 2026
81ca2a4
PYTHON-6077 Add KMS-DEBUG ssl diagnostics (temporary)
blink1073 Sep 9, 2026
3c6ec12
PYTHON-6077 Treat path UV_PYTHON as already resolved
blink1073 Sep 9, 2026
a95b78e
PYTHON-6077 Exclude 3.15t from encryption and pyopenssl variants
blink1073 Sep 9, 2026
0dc5704
PYTHON-6077 Add rich KMS-DEBUG socket diagnostics (temporary)
blink1073 Sep 9, 2026
f761273
PYTHON-6077 Use blocking SSL I/O for async KMS to fix 3.15 reset
blink1073 Sep 9, 2026
5669e4a
PYTHON-6077 Finish removing KMS-DEBUG diagnostics; harden timeout path
blink1073 Sep 9, 2026
c2b07dc
PYTHON-6077 Treat peer reset as clean close in async KMS SSL I/O
blink1073 Sep 9, 2026
ecf80e0
PYTHON-6077 Use non-blocking SSL I/O for async KMS receives
blink1073 Sep 9, 2026
99ef830
Merge remote-tracking branch 'upstream/main' into worktree-PYTHON-607…
blink1073 Sep 16, 2026
d84a016
PYTHON-6077 Drop the Python 3.15 toolchain fallback
blink1073 Sep 16, 2026
a58797f
Merge remote-tracking branch 'origin/worktree-PYTHON-6077-officially-…
blink1073 Sep 16, 2026
36e5bb4
PYTHON-6077 Select the toolchain Python with TOOLCHAIN_VERSION
blink1073 Sep 16, 2026
d407f70
PYTHON-6077 Restore FIPS UV_PYTHON and align Python setup with upstream
blink1073 Sep 16, 2026
b711bf2
PYTHON-6077 Drop out-of-scope Python setup refactors
blink1073 Sep 16, 2026
0476765
PYTHON-6077 Add missing docstrings and preserve function order in net…
blink1073 Sep 16, 2026
61942ff
PYTHON-6077 Keep the Windows platform block in its original position
blink1073 Sep 16, 2026
fa54f0f
PYTHON-6077 Harden KMS send timeout handling on Python 3.15
blink1073 Sep 16, 2026
07871f5
PYTHON-6077 Run RHEL8 variants on rhel8.10-small
blink1073 Sep 16, 2026
593c59c
PYTHON-6077 Run 3.15t encryption and PyOpenSSL tests on RHEL9.7
blink1073 Sep 16, 2026
64e0d51
PYTHON-6077 Run the RHEL9.7 encryption variants on PRs
blink1073 Sep 16, 2026
aa8625f
PYTHON-6077 Run the 3.15t encryption variants on ubuntu22
blink1073 Sep 16, 2026
a52de98
PYTHON-6077 Handle the fork DeprecationWarning in the FIPS check unde…
blink1073 Sep 16, 2026
021337f
PYTHON-6077 Suppress the fork DeprecationWarning in subprocess-spawni…
blink1073 Sep 17, 2026
1969669
PYTHON-6077 Run the 3.15t PyOpenSSL tests on ubuntu22
blink1073 Sep 17, 2026
879601b
PYTHON-6077 Run the 3.15t Encryption PyOpenSSL tests on ubuntu22
blink1073 Sep 17, 2026
e7f6c82
Merge branch 'main' into worktree-PYTHON-6077-officially-support-py315
blink1073 Sep 17, 2026
ddcdebb
Merge remote-tracking branch 'origin/worktree-PYTHON-6077-officially-…
blink1073 Sep 17, 2026
09b4da6
PYTHON-6077 Wait for the KMS send worker on cancellation too
blink1073 Sep 17, 2026
bf387a1
PYTHON-6077 Wait for the KMS send worker unconditionally and fix the …
blink1073 Sep 17, 2026
3aee647
PYTHON-6077 Match the standard wheel exactly and surface the original…
blink1073 Sep 17, 2026
e136129
PYTHON-6077 Shield the KMS send cleanup from further cancellations
blink1073 Sep 18, 2026
78b35aa
PYTHON-6077 Trim the socket-call cleanup comment
blink1073 Sep 18, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
804 changes: 584 additions & 220 deletions .evergreen/generated_configs/tasks.yml

Large diffs are not rendered by default.

144 changes: 89 additions & 55 deletions .evergreen/generated_configs/variants.yml

Large diffs are not rendered by default.

65 changes: 60 additions & 5 deletions .evergreen/scripts/generate_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -143,15 +143,15 @@ def get_encryption_expansions(encryption):
):
expansions = get_encryption_expansions(encryption)
display_name = get_variant_name(encryption, host, **expansions)
tasks = [".test-non-standard", ".test-string-query-preview"]
tasks = [".test-non-standard !.python-3.15t", ".test-string-query-preview"]
if host != "rhel8":
# Exclude PyPy (not tested with encryption on macOS/win64) and coverage tasks
# (encryption suites exceed the 60-min timeout with coverage overhead on macOS/win64).
# Also include the non-coverage companion tasks (test-non-standard-no-cov) which
# carry the "latest" server tasks without COVERAGE=1.
tasks = [
".test-non-standard !.pypy !.cov",
".test-non-standard-no-cov !.pypy",
".test-non-standard !.pypy !.cov !.python-3.15t",
".test-non-standard-no-cov !.pypy !.python-3.15t",
".test-string-query-preview",
]
variant = create_variant(
Expand All @@ -170,14 +170,45 @@ def get_encryption_expansions(encryption):
expansions = get_encryption_expansions(encryption)
display_name = get_variant_name(encryption, host, **expansions)
variant = create_variant(
[".test-non-standard"],
[".test-non-standard !.python-3.15t"],
display_name,
host=host,
expansions=expansions,
batchtime=batchtime,
tags=tags,
)
variants.append(variant)

# The 3.15t encryption tests only run on Ubuntu 22, whose OpenSSL 3.0.2 is
# needed to build the cryptography package; only MongoDB 6.0+ runs there.
host = HOSTS["ubuntu22"]
expansions = get_encryption_expansions("Encryption")
display_name = get_variant_name("Encryption", host, **expansions)
variants.append(
create_variant(
[".test-non-standard .python-3.15t !.server-4.4 !.server-5.0"],
display_name,
host=host,
expansions=expansions,
batchtime=batchtime,
tags=[*tags, "pr"],
)
)

# The 3.15t PyOpenSSL tests only run on Ubuntu 22, whose OpenSSL 3.0.2 is
# needed to build the cryptography package.
expansions = get_encryption_expansions("Encryption PyOpenSSL")
display_name = get_variant_name("Encryption PyOpenSSL", host, **expansions)
variants.append(
create_variant(
[".test-non-standard .python-3.15t"],
display_name,
host=host,
expansions=expansions,
batchtime=batchtime,
tags=[*tags, "pr"],
)
)
return variants


Expand Down Expand Up @@ -220,6 +251,8 @@ def create_compression_variants():
tasks = [
".test-standard !.server-4.4 !.server-5.0 .python-3.14",
".test-standard !.server-4.4 !.server-5.0 .python-3.14t",
".test-standard !.server-4.4 !.server-5.0 .python-3.15",
".test-standard !.server-4.4 !.server-5.0 .python-3.15t",
]
display_name = get_variant_name(f"Compression {compressor}", host)
variants.append(
Expand Down Expand Up @@ -258,7 +291,11 @@ def create_pyopenssl_variants():

for host in ["rhel8", "macos", "win64"]:
display_name = get_variant_name(base_name, host)
base_task = ".test-standard" if host == "rhel8" else ".test-standard !.pypy"
base_task = (
".test-standard !.python-3.15t"
if host == "rhel8"
else ".test-standard !.pypy !.python-3.15t"
)
# We only need to run a subset on async.
tasks = [f"{base_task} .sync", f"{base_task} .async .replica_set-noauth-ssl"]
variants.append(
Expand All @@ -271,6 +308,24 @@ def create_pyopenssl_variants():
)
)

# The 3.15t PyOpenSSL tests only run on Ubuntu 22, whose OpenSSL 3.0.2 is
# needed to build the cryptography package; only MongoDB 6.0+ runs there.
host = HOSTS["ubuntu22"]
tasks = [
".test-standard .python-3.15t !.server-4.4 !.server-5.0 .sync",
".test-standard .python-3.15t !.server-4.4 !.server-5.0 .async .replica_set-noauth-ssl",
]
variants.append(
create_variant(
tasks,
get_variant_name(base_name, host),
host=host,
expansions=expansions,
batchtime=batchtime,
tags=["pr"],
)
)

return variants


Expand Down
6 changes: 3 additions & 3 deletions .evergreen/scripts/generate_config_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
##############

ALL_VERSIONS = ["4.4", "5.0", "6.0", "7.0", "8.0", "9.0", "rapid", "latest"]
CPYTHONS = ["3.10", "3.11", "3.12", "3.13", "3.14t", "3.14"]
CPYTHONS = ["3.10", "3.11", "3.12", "3.13", "3.14t", "3.14", "3.15t", "3.15"]
PYPYS = ["pypy3.11"]
MIN_SUPPORT_VERSIONS = ["3.9", "pypy3.9", "pypy3.10"]
ALL_PYTHONS = CPYTHONS + PYPYS
Expand Down Expand Up @@ -57,7 +57,7 @@ class Host:


# Hosts with toolchains.
HOSTS["rhel8"] = Host("rhel8", "rhel87-small", "RHEL8", dict())
HOSTS["rhel8"] = Host("rhel8", "rhel8.10-small", "RHEL8", dict())

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 8.7 hosts do not have the updated toolchain yet, and 8.10 is backward-compatible

HOSTS["win64"] = Host("win64", "windows-64-vsMulti-small", "Win64", dict())
HOSTS["win-latest"] = Host("win-latest", "windows-2022-latest-small", "WinLatest", dict())
HOSTS["win32"] = Host("win32", "windows-64-vsMulti-small", "Win32", dict())
Expand Down Expand Up @@ -93,7 +93,7 @@ def create_variant_generic(
display_name: str,
*,
host: Host | str | None = None,
default_run_on="rhel87-small",
default_run_on="rhel8.10-small",
expansions: dict | None = None,
**kwargs: Any,
) -> BuildVariant:
Expand Down
8 changes: 7 additions & 1 deletion .evergreen/scripts/setup_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -290,7 +290,13 @@ def handle_test_env() -> None:
krb_conf.touch()
write_env("KRB5_CONFIG", krb_conf)
LOGGER.info("Writing keytab")
keytab = base64.b64decode(config["KEYTAB_BASE64"])
# Python 3.15 rejects unpadded base64 by default; padded=False
# accepts it, and older Pythons need the padding added explicitly.
keytab_b64 = "".join(config["KEYTAB_BASE64"].split())
try:
keytab = base64.b64decode(keytab_b64, padded=False)
except TypeError:
keytab = base64.b64decode(keytab_b64 + "=" * (-len(keytab_b64) % 4))
keytab_file = ROOT / ".evergreen/drivers.keytab"
with keytab_file.open("wb") as fid:
fid.write(keytab)
Expand Down
6 changes: 5 additions & 1 deletion .github/workflows/dist.yml
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ jobs:
# Note: the default manylinux is manylinux_2_28
run: |
python -m pip install -U pip
python -m pip install "cibuildwheel>=3.2.0,<4"
python -m pip install "cibuildwheel>=4.2.0,<5"

- name: Build wheels
env:
Expand All @@ -92,8 +92,12 @@ jobs:
ls wheelhouse/*cp312*.whl
ls wheelhouse/*cp313*.whl
ls wheelhouse/*cp314*.whl
# The free-threaded wheel's interpreter tag is also cp315, so match
# the standard wheel on its interpreter-abi boundary.
ls wheelhouse/*cp315-cp315-*.whl
# Free-threading builds:
ls wheelhouse/*cp314t*.whl
ls wheelhouse/*cp315t*.whl

- uses: actions/upload-artifact@v7
with:
Expand Down
4 changes: 4 additions & 0 deletions doc/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ Changelog
Changes in Version 4.19.0 (2026/XX/XX)
--------------------------------------

PyMongo 4.19 brings a number of changes including:

- Added support for Python 3.15.

Bug fixes
.........

Expand Down
122 changes: 48 additions & 74 deletions pymongo/network_layer.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
from typing import (
TYPE_CHECKING,
Any,
Callable,
Optional,
Union,
)
Expand Down Expand Up @@ -75,65 +76,25 @@
# the MongoDB wire protocol
async def async_socket_sendall(sock: Union[socket.socket, _sslConn], buf: bytes) -> None:
timeout = sock.gettimeout()
sock.settimeout(0.0)
loop = asyncio.get_running_loop()
try:
if _HAVE_SSL and isinstance(sock, (SSLSocket, _sslConn)):
await asyncio.wait_for(_async_socket_sendall_ssl(sock, buf, loop), timeout=timeout)
# Blocking SSL send: non-blocking send raises on a peer reset on Python 3.15.
await _async_blocking_socket_call(loop, sock.sendall, buf, timeout)
else:
await asyncio.wait_for(loop.sock_sendall(sock, buf), timeout=timeout) # type: ignore[arg-type]
# loop.sock_sendall requires a non-blocking socket.
sock.settimeout(0.0)
try:
await asyncio.wait_for(loop.sock_sendall(sock, buf), timeout=timeout) # type: ignore[arg-type]
finally:
sock.settimeout(timeout)
except asyncio.TimeoutError as exc:
# Convert the asyncio.wait_for timeout error to socket.timeout which pool.py understands.
raise socket.timeout("timed out") from exc
finally:
sock.settimeout(timeout)


if sys.platform != "win32":

async def _async_socket_sendall_ssl(
sock: Union[socket.socket, _sslConn], buf: bytes, loop: AbstractEventLoop
) -> None:
view = memoryview(buf)
sent = 0

def _is_ready(fut: Future[Any]) -> None:
if fut.done():
return
fut.set_result(None)

while sent < len(buf):
try:
sent += sock.send(view[sent:])
except BLOCKING_IO_ERRORS as exc:
fd = sock.fileno()
# Check for closed socket.
if fd == -1:
raise SSLError("Underlying socket has been closed") from None
if isinstance(exc, BLOCKING_IO_READ_ERROR):
fut = loop.create_future()
loop.add_reader(fd, _is_ready, fut)
try:
await fut
finally:
loop.remove_reader(fd)
if isinstance(exc, BLOCKING_IO_WRITE_ERROR):
fut = loop.create_future()
loop.add_writer(fd, _is_ready, fut)
try:
await fut
finally:
loop.remove_writer(fd)
if _HAVE_PYOPENSSL and isinstance(exc, BLOCKING_IO_LOOKUP_ERROR):
fut = loop.create_future()
loop.add_reader(fd, _is_ready, fut)
try:
loop.add_writer(fd, _is_ready, fut)
await fut
finally:
loop.remove_reader(fd)
loop.remove_writer(fd)

async def _async_socket_receive_ssl(
conn: _sslConn, length: int, loop: AbstractEventLoop, once: Optional[bool] = False
) -> memoryview:
Expand Down Expand Up @@ -187,28 +148,6 @@ def _is_ready(fut: Future[Any]) -> None:
else:
# The default Windows asyncio event loop does not support loop.add_reader/add_writer:
# https://docs.python.org/3/library/asyncio-platforms.html#asyncio-platform-support
# Note: In PYTHON-4493 we plan to replace this code with asyncio streams.
async def _async_socket_sendall_ssl(
sock: Union[socket.socket, _sslConn], buf: bytes, dummy: AbstractEventLoop
) -> None:
view = memoryview(buf)
total_length = len(buf)
total_sent = 0
# Backoff starts at 1ms, doubles on timeout up to 512ms, and halves on success
# down to 1ms.
backoff = 0.001
while total_sent < total_length:
try:
sent = sock.send(view[total_sent:])
except BLOCKING_IO_ERRORS:
await asyncio.sleep(backoff)
sent = 0
if sent > 0:
backoff = max(backoff / 2, 0.001)
else:
backoff = min(backoff * 2, 0.512)
total_sent += sent

async def _async_socket_receive_ssl(
conn: _sslConn, length: int, dummy: AbstractEventLoop, once: Optional[bool] = False
) -> memoryview:
Expand Down Expand Up @@ -236,11 +175,46 @@ async def _async_socket_receive_ssl(
return mv


async def _async_blocking_socket_call(
loop: AbstractEventLoop,
func: Callable[..., Any],
arg: Any,
timeout: Optional[float],
) -> Any:
"""Run a blocking socket operation in a worker thread with a timeout.

The worker cannot be interrupted, so the future is shielded from cancellation.
The call holds a default-executor thread until the socket timeout fires.
"""
inner = loop.run_in_executor(None, func, arg)
fut = asyncio.shield(inner)
try:
return await asyncio.wait_for(fut, timeout=timeout)
except (asyncio.CancelledError, asyncio.TimeoutError):
# Wait for the worker, shielded from further cancellations, so the
# caller does not close the socket under it.
while True:
try:
await asyncio.shield(inner)
break
except asyncio.CancelledError:
continue
except Exception:
break
raise
except (BrokenPipeError, ConnectionResetError, ConnectionAbortedError) as exc:
# A send to a connection the peer closed or reset raises raw on
# Python 3.15; report it as the graceful close the KMS path expects.
raise OSError("connection closed") from exc


def sendall(sock: Union[socket.socket, _sslConn], buf: bytes) -> None:
"""Send all of buf on the socket."""
sock.sendall(buf)


async def _poll_cancellation(conn: AsyncConnection) -> None:
"""Return once the connection's operation is cancelled."""
while True:
if conn.cancel_context.cancelled:
return
Expand All @@ -251,9 +225,8 @@ async def _poll_cancellation(conn: AsyncConnection) -> None:
async def async_receive_data_socket(
sock: Union[socket.socket, _sslConn], length: int
) -> memoryview:
sock_timeout = sock.gettimeout()
timeout = sock_timeout

"""Receive exactly length bytes from the socket."""
timeout = sock.gettimeout()
sock.settimeout(0.0)
loop = asyncio.get_running_loop()
try:
Expand All @@ -270,12 +243,13 @@ async def async_receive_data_socket(
except asyncio.TimeoutError as err:
raise socket.timeout("timed out") from err
finally:
sock.settimeout(sock_timeout)
sock.settimeout(timeout)


async def _async_socket_receive(
conn: socket.socket, length: int, loop: AbstractEventLoop
) -> memoryview:
"""Receive exactly length bytes from a non-blocking socket."""
mv = memoryview(bytearray(length))
bytes_read = 0
while bytes_read < length:
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
"Programming Language :: Python :: 3.15",
"Topic :: Database",
"Typing :: Typed",
]
Expand Down
10 changes: 9 additions & 1 deletion test/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -358,7 +358,15 @@ def fips_enabled(self):
if self._fips_enabled is not None:
return self._fips_enabled
try:
subprocess.run(["fips-mode-setup", "--is-enabled"], check=True)
# Python 3.15 warns on fork() in multi-threaded processes, which
# gevent's monkey-patched subprocess triggers.
with warnings.catch_warnings():
warnings.filterwarnings(
"ignore",
message=r".*use of fork\(\) may lead to deadlocks.*",
category=DeprecationWarning,
)
subprocess.run(["fips-mode-setup", "--is-enabled"], check=True)
self._fips_enabled = True
except (subprocess.SubprocessError, FileNotFoundError):
self._fips_enabled = False
Expand Down
Loading
Loading