gh-156955: Speed up csv.writer by caching the set of special characters - #157298
Merged
serhiy-storchaka merged 4 commits intoSep 13, 2026
Merged
Conversation
…aracters Cache in the dialect a 128-bit set of ASCII characters which need quoting or escaping (delimiter, quotechar, escapechar, '\r', '\n' and characters of lineterminator) and a flag whether any of them is non-ASCII. Testing a character is now one bit test instead of five comparisons and a call to PyUnicode_FindChar(). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
eendebakpt
approved these changes
Sep 11, 2026
eendebakpt
left a comment
eendebakpt
left a comment
Contributor
There was a problem hiding this comment.
Nice! A tiny suggestion to reduce the amount of comments (can be used as other places as well), but it is a matter of tase.
serhiy-storchaka
enabled auto-merge (squash)
September 13, 2026 08:00
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cache in the dialect a 128-bit set of ASCII characters that need quoting or escaping (delimiter, quotechar, escapechar,
\r,\nand the characters of lineterminator), and a flag whether any of them is non-ASCII. Testing a character is now one bit test instead of five comparisons and a call toPyUnicode_FindChar(); the old chain is only used for non-ASCII characters when the dialect has a non-ASCII special character.Alternative to #156956 (which only caches the maximum character of lineterminator). Results with the same interleaved benchmark (release build, non-PGO), median of 15 rounds:
Fields of 2000 characters are written 4.6x faster, of 20000 characters 5.9x faster.
Tests now cover non-ASCII delimiter, quotechar, escapechar and lineterminator in the writer.
🤖 Generated with Claude Code