Skip to content

~150x-300x faster downloads with numerapi#127

Merged
ndharasz merged 1 commit into
numerai:masterfrom
jeethu:download-chunk-size
Jun 2, 2026
Merged

~150x-300x faster downloads with numerapi#127
ndharasz merged 1 commit into
numerai:masterfrom
jeethu:download-chunk-size

Conversation

@jeethu
Copy link
Copy Markdown
Contributor

@jeethu jeethu commented Jun 2, 2026

With this minimal patch, download speeds went from ~30-60 KB/s to ~9-10 MB/s.

Network engineers hate this one weird trick.

IPv4 MTU is ~1500 bytes.
IPv6 minimum packet size is 1280 bytes.
Chunk size of 1024 was bad even back in the dialup days.

@ndharasz ndharasz merged commit c091702 into numerai:master Jun 2, 2026
2 checks passed
@jeethu jeethu deleted the download-chunk-size branch June 2, 2026 21:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants