Skip to content

PIL._binary: Use int.to_bytes() for faster conversion - #9981

Open
akx wants to merge 1 commit into
python-pillow:mainfrom
akx:faster-binary-helpers
Open

PIL._binary: Use int.to_bytes() for faster conversion#9981
akx wants to merge 1 commit into
python-pillow:mainfrom
akx:faster-binary-helpers

Conversation

@akx

@akx akx commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

int.to_bytes() has been a thing since Python 3.2, and it's faster than struct.pack having to parse that spec and all that on every invocation. This micro-benchmarks to be 1.2x to 1.5x faster on my machine for essentially free.

Fun fact

There's an upcoming CPython 3.16 optimization that will further help o8() memory-wise (since the int.to_bytes(..., 1) call will end up just returning one of the singleton byteses).

Fun related fact

We could borrow that idea, namely something like

 from struct import unpack_from
 
+_o8_bytes = tuple(bytes(range(256))[n : n + 1] for n in range(256))
+
 
 def i8(c: bytes) -> int:
     return c[0]
 
 
 def o8(i: int) -> bytes:
-    return (i & 255).to_bytes()
+    return _o8_bytes[i & 255]

but it felt a little too much. It would save about 42 bytes of allocations for every o8() call before 3.16 lands, as well as some time (microbenchmarked to be 2.6x faster), but it's a bit weird - I suspect better gains would be had by looking at the call sites for o8() to see if they really need to pack a single byte at a time. (The weird construction for the tuple ensures we do actually get references to the immortal singleton byteses, not new allocations.)

@akx
akx marked this pull request as ready for review September 10, 2026 13:21
@akx

akx commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

I suspect our benchmark suite isn't really testing the write side of plugins that use o8() (BMP, DDS, FLI, GIF, GIMP Gradients, ICO, JPEG, Palm, PCX, PNG, PPM, QOI, SGI, TGA, TIFF, XPM, XVThumb), so CodSpeed doesn't say much about this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants