Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 41 additions & 4 deletions .github/workflows/ci-cd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -546,10 +546,15 @@ jobs:
- os: ubuntu-latest
qemu: s390x
musl: musllinux
- os: ubuntu-latest
# armv7l builds on aarch64 hosts. We still register QEMU so
# binfmt picks up the 32-bit ARM userspace handler regardless of
# whether the host kernel has CONFIG_COMPAT enabled. Even with
# emulation, aarch64-on-aarch64 hosting beats x86_64 by a wide
# margin.
- os: ubuntu-24.04-arm
qemu: armv7l
musl: ""
- os: ubuntu-latest
- os: ubuntu-24.04-arm
qemu: armv7l
musl: musllinux
- os: ubuntu-latest
Expand Down Expand Up @@ -577,6 +582,9 @@ jobs:
# Build emulated architectures only if QEMU is set,
# use default "auto" otherwise
echo "CIBW_ARCHS_LINUX=${{ matrix.qemu }}" >> $GITHUB_ENV
# Override pyproject.toml's `build[uv]`: the pypa odd-arch
# manylinux/musllinux containers do not ship `uv` preinstalled.
echo "CIBW_BUILD_FRONTEND=build" >> $GITHUB_ENV
fi
shell: bash
- name: Setup Python
Expand All @@ -600,8 +608,11 @@ jobs:
with:
# `build-frontend = "build[uv]"` (pyproject.toml) requires uv to be
# available on the runner for Windows and macOS. Installing
# cibuildwheel with the `uv` extra bundles uv with it; Linux
# already has uv inside the manylinux/musllinux container.
# cibuildwheel with the `uv` extra bundles uv with it; the
# tested-arch manylinux/musllinux containers also ship uv
# preinstalled. The odd-arch containers do not, so the
# `Prepare emulation` step above sets `CIBW_BUILD_FRONTEND=build`
# for those QEMU matrix cells.
extras: uv
env:
CIBW_SKIP: pp* ${{ matrix.musl == 'musllinux' && '*manylinux*' || '*musllinux*' }}
Expand Down Expand Up @@ -652,7 +663,29 @@ jobs:
- name: Collected dists
run: |
tree dist
- name: Check whether the GitHub Release already exists
# Allows re-running the deploy job after a partial failure (e.g. PyPI
# upload error) without the Make Release step failing with HTTP 422
# because the tag/release was created on a prior attempt. Treat
# only the literal `release not found` reply as "does not exist";
# other failures (auth, rate-limit, network) re-raise so the job
# fails loudly instead of falling through to Make Release.
id: gh-release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAG: ${{ github.ref_name }}
run: |
if gh release view "${TAG}" --repo "${GITHUB_REPOSITORY}" \
>/dev/null 2>err; then
echo 'exists=true' >> "${GITHUB_OUTPUT}"
elif grep -qx 'release not found' err; then
echo 'exists=false' >> "${GITHUB_OUTPUT}"
else
cat err >&2
exit 1
fi
- name: Make Release
if: steps.gh-release.outputs.exists != 'true'
uses: aio-libs/create-release@v1.6.6
with:
changes_file: CHANGES.rst
Expand All @@ -668,6 +701,10 @@ jobs:
- name: >-
Publish 🐍📦 to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
# Allow re-running the deploy job after a partial PyPI upload
# without failing on dists that were already published.
skip-existing: true

- name: Sign the dists with Sigstore
uses: sigstore/gh-action-sigstore-python@v3.3.0
Expand Down
2 changes: 2 additions & 0 deletions CHANGES/12452.feature.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Added :attr:`~aiohttp.ClientResponse.output_size` and
:attr:`~aiohttp.ClientResponse.upload_complete` -- by :user:`Dreamsorcerer`.
4 changes: 4 additions & 0 deletions CHANGES/12647.contrib.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Override ``CIBW_BUILD_FRONTEND=build`` on the QEMU-emulated odd-arch wheel
jobs so cibuildwheel falls back to plain pip, because the pypa
``manylinux``/``musllinux`` containers for those arches do not ship ``uv``
preinstalled -- by :user:`bdraco`.
5 changes: 5 additions & 0 deletions CHANGES/12651.contrib.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Allowed re-running the ``deploy`` job in ``.github/workflows/ci-cd.yml``
after a partial release failure: the ``Make Release`` step now skips
when the GitHub Release already exists, and the PyPI publish step uses
``skip-existing`` so dists that were already uploaded on a prior
attempt do not break the retry -- by :user:`bdraco`.
4 changes: 4 additions & 0 deletions CHANGES/12655.contrib.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Switched the armv7l wheel builds onto GitHub's hosted ARM runners. The
32-bit ARM build still runs under QEMU, but the host is now aarch64
rather than x86_64, so the emulation overhead drops sharply
-- by :user:`bdraco`.
54 changes: 49 additions & 5 deletions aiohttp/client_reqrep.py
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,9 @@ class ClientResponse(HeadersMixin):
_resolve_charset: Callable[["ClientResponse", bytes], str] = lambda *_: "utf-8"

__writer: asyncio.Task[None] | None = None
_stream_writer: AbstractStreamWriter | None = None
_output_size: int = 0
_upload_complete: asyncio.Future[None] | None = None

def __init__(
self,
Expand All @@ -226,6 +229,7 @@ def __init__(
session: "ClientSession | None",
request_headers: CIMultiDict[str],
original_url: URL,
stream_writer: AbstractStreamWriter,
**kwargs: object,
) -> None:
# kwargs exists so authors of subclasses should expect to pass through unknown
Expand All @@ -240,7 +244,10 @@ def __init__(

self._real_url = url
self._url = url.with_fragment(None) if url.raw_fragment else url
if writer is not None:
if writer is None: # Request already sent
self._output_size = stream_writer.output_size
else:
self._stream_writer = stream_writer
self._writer = writer
if continue100 is not None:
self._continue = continue100
Expand All @@ -261,6 +268,11 @@ def __init__(

def __reset_writer(self, _: object = None) -> None:
self.__writer = None
if self._stream_writer is not None:
self._output_size = self._stream_writer.output_size
self._stream_writer = None
if self._upload_complete is not None and not self._upload_complete.done():
self._upload_complete.set_result(None)

@property
def _writer(self) -> asyncio.Task[None] | None:
Expand All @@ -281,10 +293,29 @@ def _writer(self, writer: asyncio.Task[None] | None) -> None:
return
if writer.done():
# The writer is already done, so we can clear it immediately.
self.__writer = None
self.__reset_writer()
else:
writer.add_done_callback(self.__reset_writer)

@property
def output_size(self) -> int:
"""Number of bytes sent for this request."""
if self._stream_writer is not None:
return self._stream_writer.output_size
return self._output_size

@property
def upload_complete(self) -> "asyncio.Future[None]":
"""Future set when the request body has been fully sent.

Already done when the request had no body or was written eagerly.
"""
if self._upload_complete is None:
self._upload_complete = self._loop.create_future()
if self._stream_writer is None: # upload already finished
self._upload_complete.set_result(None)
return self._upload_complete

@property
def cookies(self) -> SimpleCookie:
if self._cookies is None:
Expand Down Expand Up @@ -558,6 +589,9 @@ async def _wait_released(self) -> None:
def _cleanup_writer(self) -> None:
if self.__writer is not None:
self.__writer.cancel()
if self._stream_writer is not None:
self._output_size = self._stream_writer.output_size
self._stream_writer = None
self._session = None

def _notify_content(self) -> None:
Expand Down Expand Up @@ -800,7 +834,11 @@ def _update_headers(self, headers: CIMultiDict[str]) -> None:
self.headers[hdrs.HOST] = headers.pop(hdrs.HOST, host)
self.headers.extend(headers)

def _create_response(self, task: asyncio.Task[None] | None) -> ClientResponse:
def _create_response(
self,
task: asyncio.Task[None] | None,
stream_writer: AbstractStreamWriter,
) -> ClientResponse:
return self.response_class(
self.method,
self.original_url,
Expand All @@ -812,6 +850,7 @@ def _create_response(self, task: asyncio.Task[None] | None) -> ClientResponse:
session=None,
request_headers=self.headers,
original_url=self.original_url,
stream_writer=stream_writer,
)

def _create_writer(self, protocol: BaseProtocol) -> StreamWriter:
Expand Down Expand Up @@ -885,7 +924,7 @@ async def _send(self, conn: "Connection") -> ClientResponse:
protocol.start_timeout()
writer.set_eof()
task = None
self._response = self._create_response(task)
self._response = self._create_response(task, stream_writer=writer)
return self._response

async def _write_bytes(
Expand Down Expand Up @@ -1261,7 +1300,11 @@ def _update_proxy(
self.proxy = proxy
self.proxy_headers = proxy_headers

def _create_response(self, task: asyncio.Task[None] | None) -> ClientResponse:
def _create_response(
self,
task: asyncio.Task[None] | None,
stream_writer: AbstractStreamWriter,
) -> ClientResponse:
return self.response_class(
self.method,
self.original_url,
Expand All @@ -1273,6 +1316,7 @@ def _create_response(self, task: asyncio.Task[None] | None) -> ClientResponse:
session=self._session,
request_headers=self.headers,
original_url=self.original_url,
stream_writer=stream_writer,
)

def _create_writer(self, protocol: BaseProtocol) -> StreamWriter:
Expand Down
24 changes: 24 additions & 0 deletions docs/client_reference.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1538,6 +1538,30 @@ Response object

.. versionadded:: 3.2

.. attribute:: output_size

Number of bytes sent for this request.

Pair with :attr:`upload_complete` to display upload progress::

async with session.post(url, data=mpwriter) as resp:
while not resp.upload_complete.done():
print(f"uploaded {resp.output_size} bytes")
await asyncio.sleep(0.5)
print(f"upload complete: {resp.output_size} bytes")

.. versionadded:: 3.14

.. attribute:: upload_complete

An :class:`asyncio.Future` set when the request body has been fully sent.

Use ``await resp.upload_complete`` to block until the upload finishes, or
``resp.upload_complete.done()`` to poll from a progress-sampling loop
(see :attr:`output_size`).

.. versionadded:: 3.14

.. attribute:: content_type

Read-only property with *content* part of *Content-Type* header.
Expand Down
5 changes: 5 additions & 0 deletions docs/spelling_wordlist.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
aarch
abc
ABI
addons
Expand Down Expand Up @@ -68,6 +69,7 @@ charset
charsetdetect
chunked
chunking
cibuildwheel
CIMultiDict
ClientSession
cls
Expand Down Expand Up @@ -108,6 +110,7 @@ Dev
dict
Dict
Discord
dists
django
Django
dns
Expand Down Expand Up @@ -264,6 +267,8 @@ py
pydantic
pyenv
pyflakes
pypa
PyPI
pyright
pytest
Pytest
Expand Down
2 changes: 1 addition & 1 deletion requirements/base-ft.txt
Original file line number Diff line number Diff line change
Expand Up @@ -45,5 +45,5 @@ typing-extensions==4.15.0 ; python_version < "3.13"
# -r requirements/runtime-deps.in
# aiosignal
# multidict
yarl==1.22.0
yarl==1.24.2
# via -r requirements/runtime-deps.in
2 changes: 1 addition & 1 deletion requirements/base.txt
Original file line number Diff line number Diff line change
Expand Up @@ -47,5 +47,5 @@ typing-extensions==4.15.0 ; python_version < "3.13"
# multidict
uvloop==0.22.1 ; platform_system != "Windows" and implementation_name == "cpython"
# via -r requirements/base.in
yarl==1.22.0
yarl==1.24.2
# via -r requirements/runtime-deps.in
6 changes: 2 additions & 4 deletions requirements/constraints.txt
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,7 @@ async-timeout==5.0.1 ; python_version < "3.11"
# aiohttp
# valkey
attrs==26.1.0
# via
# aiohttp
# myst-parser
# via aiohttp
babel==2.18.0
# via sphinx
backports-asyncio-runner==1.2.0
Expand Down Expand Up @@ -334,7 +332,7 @@ wait-for-it==2.3.0
# via -r requirements/test-common.in
wheel==0.47.0
# via pip-tools
yarl==1.22.0
yarl==1.24.2
# via
# -r requirements/runtime-deps.in
# aiohttp
Expand Down
6 changes: 2 additions & 4 deletions requirements/dev.txt
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,7 @@ async-timeout==5.0.1 ; python_version < "3.11"
# aiohttp
# valkey
attrs==26.1.0
# via
# aiohttp
# myst-parser
# via aiohttp
babel==2.18.0
# via sphinx
backports-asyncio-runner==1.2.0
Expand Down Expand Up @@ -324,7 +322,7 @@ wait-for-it==2.3.0
# via -r requirements/test-common.in
wheel==0.47.0
# via pip-tools
yarl==1.22.0
yarl==1.24.2
# via
# -r requirements/runtime-deps.in
# aiohttp
Expand Down
2 changes: 1 addition & 1 deletion requirements/lint.txt
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ valkey==6.1.1
# via -r requirements/lint.in
virtualenv==21.3.3
# via pre-commit
yarl==1.23.0
yarl==1.24.2
# via aiohttp
zlib-ng==1.0.0
# via -r requirements/lint.in
2 changes: 1 addition & 1 deletion requirements/runtime-deps.txt
Original file line number Diff line number Diff line change
Expand Up @@ -41,5 +41,5 @@ typing-extensions==4.15.0 ; python_version < "3.13"
# -r requirements/runtime-deps.in
# aiosignal
# multidict
yarl==1.22.0
yarl==1.24.2
# via -r requirements/runtime-deps.in
2 changes: 1 addition & 1 deletion requirements/test-common.txt
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ typing-inspection==0.4.2
# via pydantic
wait-for-it==2.3.0
# via -r requirements/test-common.in
yarl==1.23.0
yarl==1.24.2
# via aiohttp
zlib-ng==1.0.0
# via -r requirements/test-common.in
2 changes: 1 addition & 1 deletion requirements/test-ft.txt
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ typing-inspection==0.4.2
# via pydantic
wait-for-it==2.3.0
# via -r requirements/test-common.in
yarl==1.22.0
yarl==1.24.2
# via
# -r requirements/runtime-deps.in
# aiohttp
Expand Down
2 changes: 1 addition & 1 deletion requirements/test.txt
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ uvloop==0.22.1 ; platform_system != "Windows" and implementation_name == "cpytho
# via -r requirements/base.in
wait-for-it==2.3.0
# via -r requirements/test-common.in
yarl==1.22.0
yarl==1.24.2
# via
# -r requirements/runtime-deps.in
# aiohttp
Expand Down
Loading
Loading