Add R01, before your first line

[tamnd]

The first of the runtime lessons, and a change of subject. Everything up to here has been about what happens when your code runs. This one is about everything that has already happened by the time it starts.

How it is put together

Eight cells, and every one of them starts a child interpreter and asks it a question rather than looking at the notebook it is running in. That is not a stylistic choice. The notebook finished starting up long ago and has since imported hundreds of modules that have nothing to do with the subject, so it cannot be a witness about its own startup. All eight cells check whether they can start a process at all and say so rather than failing, which is what makes the lesson read through in a browser tab.

What is in it

The inventory. A child counts its own sys.modules and splits it by where each module came from: C compiled into the binary, Python bytecode frozen into the binary, or a path, which means somebody opened a file. With -S there are 21 modules and not one of them was read from a file. The cell also names the ones that did come from a file on a normal start, which on most installs is whatever site brought in for you rather than anything the interpreter needed.

The two halves. Prose, with the chain from pymain_main through pymain_init and Py_InitializeFromConfig to Py_RunMain. Core startup builds the runtime and the built in types with no import system, no sys.path and no codecs. Main startup is everything that needs those, including working out sys.path. That split is why a configuration mistake comes out as a fatal error with a plain C string: at the moment it is noticed there is nothing to raise. Then -X importtime in a child, which prints exactly the imports that happen before your first line and nothing else.

The configuration. Seven children with different combinations of PYTHONOPTIMIZE, -O, -OO and -E, all reporting sys.flags.optimize. The natural guess is that the command line beats the environment because it is nearer, and it is wrong. The helper that reads a flag out of the environment is if (*flag < value) *flag = value;, which raises a floor and never lowers anything, and -O on the command line increments rather than assigns. So the answer is the highest number anybody asked for. PYTHONOPTIMIZE=2 left in a shell profile is not undone by passing -O today, and nothing tells you. -E is the only way out, and it works by deleting the environment rather than outranking it.

The path. sys.path is not computed in C. It is computed by Modules/getpath.py, compiled to bytecode when CPython is built, stored in the binary as a marshalled blob, unmarshalled during main startup and evaluated. It cannot import anything, because sys.path is what it exists to produce, so it is handed eleven C functions to stand in for os.path: abspath, basename, dirname, hassuffix, isabs, isdir, isfile, isxfile, joinpath, readlines, realpath. The cell prints the whole path configuration with an exists check next to every entry, which makes the pythonXY.zip that is not there stand out. That entry is deliberate, not leftover.

The front of the path. sys.path[0] compared across -c, a script, -m and -P. It is pushed on after startup is finished, by the code that is about to hand control to your program, and it is the directory of your script or the current directory. That is the complete mechanism behind a file called random.py next to your script shadowing the standard library. -P does not remove the entry, it never adds it, so sys.path[0] becomes that zip file instead.

The cost. Ten timed starts of an interpreter that runs nothing, with and without site.

The recordings

Two Tier 1 recordings, and the first release against debug pair in the book. Same program, same image pipeline, one build configured with --with-pydebug.

The debug build starts in 56.5 ms against 26.5 ms. The easy explanation is the assertions and the reference count bookkeeping, and the easy explanation is mostly wrong. Release reports 17 frozen modules and 0 from a file. Debug reports 3 frozen and 14 from a file. That is not the same work done more slowly, it is different work. A debug build turns frozen modules off in the defaults, one #ifdef, so that somebody debugging CPython steps through the real Lib/os.py rather than through bytecode baked in at build time. The import bill goes from 11.6 ms to 30.8 ms with it, and -X frozen_modules=on puts it back.

Also

A new glossary group for startup and shutdown with three terms: two phase initialisation, path configuration, safe path. GLOSSARY.md is now 216 terms. Six diagrams, sixteen citations, both READMEs updated, CLAIMS.md is 564 claims across 72 lessons.

just check and just versions are green locally, 167 declared and 181 noted across 72 notebooks, and the browser probe has been rerun. All eight R01 cells report that the runtime cannot start another interpreter and carry on.

Part of #27.