[slozier]

Use importlib for __import__. Includes fast paths written in C# so performance should be similar to what we had before. The legacy mechanism is still available with the (debug-only) -X NoImportLib option.

Breaking changes:

• you can no longer import mixed code (like a ruby file)
• the PAL is no longer used (will be adding a PalImporter to re-enable this as a follow-up PR)
• a module file added to a directory after an import from it may not be found until importlib.invalidate_caches() is called, like CPython
• because we're using importlib._bootstrap, custom importers now follow CPython semantics (find_spec is preferred over find_module, etc.)
• the console prints an ImportError if site can't be found (use -S to skip)

Follow up items:

• PalImporter (in progress)
• enable more tests (in progress)
• clean up traceback frames
• remove IronPython specific machinery from ImportModuleWithImportLib:
  • investigate setting ShowCls at the import statement, like __future__
  • refresh stale .NET names after clr.AddReference outside of __import__
• remove legacy importer

[BCSharp]

I see that this is work in progress, but here are my findings so far after playing with it for a while.

[BCSharp]

Consider this:

>>> import clr
>>> g = clr.GetCurrentRuntime().Globals
>>> import sys
>>> sys.modules['blocked_name'] = None
>>> import blocked_name
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 2222, in _gcd_import
ImportError: import of blocked_name halted; None in sys.modules
>>> g.blocked_name = 42               
>>> import blocked_name
>>> sys.modules['blocked_name']
42

Looks like a regression. ImportError is expected in both cases.

[BCSharp]

# assume packages p, p.a, p.a.b already exist
import p.a.b
import sys
sys.modules['p.a'] = None
__import__("a.b", {"__package__": "p"}, None, None, 1)
# CPython: None (value of sys.modules['p.a'])
# IronPython: ImportError: No module named 'a.b'

Perhaps setting top = module when firstDot == -1?

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

[slozier]

Ran into a complication for which I'm undecided. Using importlib we get PEP 420 for free, but this means that if we have a folder called System and we do import System we'll get the namespace package instead of the .NET namespace. So the part I'm undecided about is precedence - should we expect the .NET namespace or the package namespace...

[BCSharp]

Ran into a complication for which I'm undecided. Using importlib we get PEP 420 for free, but this means that if we have a folder called System and we do import System we'll get the namespace package instead of the .NET namespace. So the part I'm undecided about is precedence - should we expect the .NET namespace or the package namespace...

I had a similar dilemma with codecs: in case of a name conflict, prefer .NET codecs or Python installed codecs? In the end I thought it's better to give precedence to .NET, since IronPython is a .NET language. Perhaps with an option to override in some obscure scenarios.

In case of import, I think it is less problematic as conflicts should be rare. The convention for .NET is names capitalized, and for Python 3 it is all lowercase.