Skip to content

Faster downloads: 2 KB base64 chunks (#133) - #180

Merged
bdbarnett merged 1 commit into
mainfrom
perf/download-133
Oct 29, 2026
Merged

bdbarnett merged 1 commit into
mainfrom
perf/download-133

Conversation

@bdbarnett

Copy link
Copy Markdown
Collaborator

Downloading a file from a board was 4 to 6 times slower than uploading it. Each 256-byte chunk of a download costs one round trip with a printed reply, and on CircuitPython a reply costs far more than a silent write, so 80 round trips for 20 KB dominated. Chunk size was the lever; the reply was also up to 4x the data as repr of binary.

This reads in 2048-byte chunks and has the board send each chunk as base64 (binascii), checked with a strict character pattern and the existing length check. A board without binascii keeps the old 256-byte repr path, and a board that raises MemoryError gets the chunk halved down to 256.

Measured on an ESP32-S3 running CircuitPython 11 over USB serial (COM17), same board and transport, mpftp put/cp --no-verify, whole command including about 5 s of start-up, files compared with cmp afterwards:

size put get before get after
4 KB 6.9 s 17.1 s 8.7 s
20 KB 15.1 s 64.5 s 17.4 s
100 KB 53.5 s 309.5 s 55.1 s

A 20 KB cp with the default verify, which reads the file a second time on this board, went from 130 s to 67 s.

Tests: python -m unittest cli.tests.test_read_file_checked (new cases for chunk count, no-binascii fallback, small heap and base64 noise).

Closes #133.

@bdbarnett
bdbarnett merged commit 3922b5b into main Oct 29, 2026
8 checks passed
@bdbarnett
bdbarnett deleted the perf/download-133 branch October 29, 2026 05:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Desktop: downloading from the board is much slower than uploading

1 participant