Skip to content

Add R06, the C API tiers - #185

Merged
tamnd merged 1 commit into
mainfrom
r06-the-c-api-tiers
Sep 6, 2026
Merged

Add R06, the C API tiers#185
tamnd merged 1 commit into
mainfrom
r06-the-c-api-tiers

Conversation

@tamnd

@tamnd tamnd commented Sep 6, 2026

Copy link
Copy Markdown
Owner

R06 is the sixth runtime lesson and the first one that looks at CPython from the C side rather than from inside Python. It answers a question that comes up the moment anybody opens Python.h: which of these functions am I actually allowed to call, and what happens if I call one I am not.

The answer is three directories and two macros. Include/ is open to any extension. Include/cpython/ needs Py_LIMITED_API to be undefined, and the mechanism is nothing more than an #ifndef around an #include at the bottom of each public header. Include/internal/ starts nearly every file with three lines that stop the compiler with an error unless you define Py_BUILD_CORE. On 3.15 that third directory is 148 files and 42552 lines, which is more than the other two put together.

The lesson walks the Py_LIMITED_API guard as a preprocessor stack, so the count of what a limited build may call is measured rather than asserted: 580 functions visible in the public headers, 186 taken away by the guard, and 974 in the other two directories it never opens. It then makes the point that the functions are not the expensive part. The struct layouts go too, and with no fields to read Py_TYPE becomes a function call and Py_DECREF becomes a call to _Py_DecRef.

The naming convention gets cross tabulated against the directories, because people assume the two are the same thing and they nearly are. The seventeen underscore names sitting in the public tier all have one explanation: a private name has to be exported when a public macro expands to it, which is why _Py_Dealloc is a public symbol.

Then the part that changes how you read all of it. None of the tiers survive into the binary. Eight names from all three tiers go to ctypes.pythonapi and every one of them resolves, and _PyDict_SizeOf gets called by hand and returns exactly what dict.__sizeof__ returns, 16 bytes short of sys.getsizeof because that adds the collector header.

Two Tier 1 recordings settle whether that is an accident. It is not. Inside the internal headers, 530 names are spelled PyAPI_FUNC and 493 of them resolve on the release build, against 757 spelled plain extern of which 3 do. Both spellings sit in the same files, often two lines apart, and 168 comments name which bundled shared extension needs each export. The free threaded run finds six more, which are the ones behind Py_GIL_DISABLED, so the missing names were a build flag rather than a mystery.

The last cell hands off to R07 by looking at what a built extension does carry, which is its file name. Three suffixes on 3.14 and six on 3.15, and the new ones end in abi3t.

Half of this wants to read the headers, and a browser tab has none. Those cells sit behind one flag set in the first of them and say so out loud rather than printing nothing, the same shape R05 used. The other half only needs ctypes, which works everywhere, so all eleven probed cells run under Pyodide.

Six diagrams, four glossary terms, five differs= notes checked against a real 3.14 run, and rows in both READMEs.

The sixth runtime lesson, and the first one that looks at CPython from the
C side rather than from inside Python.

The C API is three directories and two macros. `Include/` is open to any
extension, `Include/cpython/` needs `Py_LIMITED_API` to be undefined, and
`Include/internal/` starts nearly every file with three lines that stop
the compiler unless you define `Py_BUILD_CORE`. More than half the header
lines are in that third directory.

The lesson walks the `Py_LIMITED_API` guard as a preprocessor stack to
count what a limited build may call, prints the `#error` that guards 140
of the 148 internal headers, cross tabulates the three naming conventions
against the three directories, and then shows that none of it survives
the build: every tier resolves through `ctypes.pythonapi`, and calling
`_PyDict_SizeOf` by hand gives the same number `dict.__sizeof__` does.

Two Tier 1 recordings put the split on record. Inside the internal
headers, 93 percent of the names spelled `PyAPI_FUNC` resolve against 0.4
percent of the ones spelled plain `extern`, with 168 comments naming
which bundled shared extension needs each export. The free threaded run
finds six more, which are the ones behind `Py_GIL_DISABLED`.

Half the lesson wants to read the headers and a browser tab does not have
them, so the header cells sit behind one flag set in the first of them
and say so rather than printing nothing. The other half only needs
ctypes, which works everywhere, so eleven of eleven cells run under
Pyodide.

Six diagrams, four glossary terms, rows in both READMEs.
@tamnd tamnd added this to the M8 Concurrency and runtime milestone Sep 6, 2026
@tamnd tamnd added kind/lesson A chapter: prose, notebook, experiments, boss fight area/runtime Startup, shutdown, import, extension modules and the C API labels Sep 6, 2026
@tamnd
tamnd merged commit 0c1d755 into main Sep 6, 2026
16 checks passed
@tamnd
tamnd deleted the r06-the-c-api-tiers branch September 6, 2026 09:41
@tamnd tamnd mentioned this pull request Sep 6, 2026
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/runtime Startup, shutdown, import, extension modules and the C API kind/lesson A chapter: prose, notebook, experiments, boss fight

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant