gh-149504: Fix re-entrancy bug when .pth/.start file invokes site.addsitedir()

[warsaw]

Fix site.addsitedir() to allow re-entrant calls from within startup files. Previously, a .pth file containing an import line that called site.addsitedir() (or a .start entry point doing the same) could crash with RuntimeError: dictionary changed size during iteration during site initialization, breaking tools such as uv run --with.

mkdir -p /tmp/site-repro/root /tmp/site-repro/overlay
printf 'import site; site.addsitedir("/tmp/site-repro/overlay")\n' > /tmp/site-repro/root/reenter.pth
./python.exe -S -c 'import site; site.addsitedir("/tmp/site-repro/root"); print("done")'

prints "done" for me.

uv run --python /Users/jelle/py/cpython/python.exe --with iniconfig python -c 'import iniconfig'

does not crash.

The implementation is more comprehensive than #149516 and is more inline with a suggestion @ncoghlan made on the original PR. Thanks @zsol for providing your PR. @JelleZijlstra please take a look.

• Issue: [3.15] Re-entrant calls to site.addsitedir() can fail (affecting uv run) #149504

[zsol]

FWIW, I reviewed the changes and checked that uv run --with ... works fine with these. Encapsulating all the state into a class is nice :) I'll close #149516 in favor of this

[JelleZijlstra]

I'll try to take a look if you want more reviews but I don't really know site.py. Thanks for working on this!

[warsaw]

I'll try to take a look if you want more reviews but I don't really know site.py. Thanks for working on this!

That's okay. I just tagged you to check that the uv case works with this branch, but @zsol confirmed it fixed the problem (thanks!).

[miss-islington-app]

Thanks @warsaw for the PR 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

[miss-islington-app]

Sorry, @warsaw, I could not cleanly backport this to 3.15 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker b162307d7f216e87976d76c9b8f4a932961cb2d4 3.15

[bedevere-app]

GH-149799 is a backport of this pull request to the 3.15 branch.