gh-156939: Document that PyBytesObject ends with a NUL byte

[vstinner]

• Issue: C API: Detect buffer overflow in PyBytesWriter #156939

[read-the-docs-community]

Documentation build overview

📚 cpython-previews | 🛠️ Build #34569933 | 📁 Comparing 1544e00 against main (9398655)

  🔍 Preview build  

57 files changed · + 8 added · ± 48 modified · - 1 deleted

+ Added

• builtins/constants.html
• builtins/exceptions.html
• builtins/functions.html
• builtins/index.html
• builtins/stdtypes.html
• builtins/threadsafety.html
• builtins/time-complexity.html
• library/tkinter.dialogs.html

± Modified

• glossary.html
• index.html
• c-api/bytearray.html
• c-api/bytes.html
• c-api/complex.html
• c-api/typeobj.html
• extending/index.html
• faq/general.html
• howto/functional.html
• library/bz2.html
• and 38 more...

- Deleted

• library/dialog.html

[vstinner]

The trailing NUL byte property is already documented in:

• PyBytes_AsString: "The last byte in the buffer is always null, regardless of whether there are any other null bytes."
• PyBytes_AsStringAndSize: "The buffer refers to an internal buffer of obj, which includes an additional null byte at the end (not counted in length)."

[ngoldbaum]

Seems worth mentioning to me!

[encukou]

I'd rather keep this in the function docs. Or if it needs to be here, then as a CPython implementation detail. (In another implementation, PyBytes_AsString might realloc to add the null byte.)

[vstinner]

I'd rather keep this in the function docs. Or if it needs to be here, then as a CPython implementation detail. (In another implementation, PyBytes_AsString might realloc to add the null byte.)

Ah ok, I see you point. It sounds inefficient to me to duplicate the whole string in PyBytes_AsString() to add an extra null byte. It would be more efficient to always allocate that byte. But I take your point and I modified to note to document it as a CPython implementation detail.

[encukou]

It sounds inefficient to me to duplicate the whole string in PyBytes_AsString() to add an extra null byte.

In CPython, yes. I'm thinking about Jython-ish implementations where bytes could often reuse storage of a “native” object.

[vstinner]

Oh, in fact bytearray also has this trailing null byte. I update my PR to add a similar implementation detail note to bytearray.

[vstinner]

I updated the branch to retrieve a fix for the Docs CI.

[miss-islington-app]

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14, 3.15.
🐍🍒⛏🤖

[bedevere-app]

GH-157550 is a backport of this pull request to the 3.15 branch.

[bedevere-app]

GH-157551 is a backport of this pull request to the 3.14 branch.

[bedevere-app]

GH-157552 is a backport of this pull request to the 3.13 branch.

[vstinner]

Thanks for your help on that doc change @encukou.

@ngoldbaum spotted that RustPython omits this trailing null bytes in its C API! RustPython/RustPython#8685