When an import failed with a stack overflow RecursionError, releasing the import locks on the error path could itself raise RecursionError, because the cleanup runs in a new C frame. This left the global import lock or a module lock held forever, and later imports in other threads blocked indefinitely.
_imp.acquire_lock() now raises RecursionError when there is not enough stack left to release the lock. importlib calls it before taking any import lock, so this covers the per-module locks too. The module lock weakref callback skips cleanup in that case instead of
reporting an unraisable exception.
- Unify `_Py_ext_module_loader_info_init*` functions. The “core” one was entirely unused, and the difference between “builtin” and “extension” can be expressed in arguments.
- Consolidate getting & checking the module name from spec. (This improves error message wording for `imp.create_builtin` with empty name.)
- Allow non-ASCII names.
Core modules (`sys` and `builtins`) still don't go through the routine.
Co-authored-by: Itamar Oren <itamarost@gmail.com>
A single-phase init function that creates its module using only the
last component of a dotted name (as pybind11 does for submodules) got
a module whose __name__ was the short name. Set the package context
while calling the init function, as the dynamic loader does, so the
module gets the full name from the spec.
Destructors can run after finalization has cleared the interpreter's
lazy_modules set and lazy_pending_submodules dict. Declaring or
resolving a lazy import from one of them passed NULL to PySet_Add(),
PySet_Discard() or the pending-submodules lookup.
Treat the bookkeeping as a no-op once the state is gone, as the other
readers of this state already do.
* gh-158140: Resolve lazy siblings through the pending-child registry
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: T. Wouters <thomas@python.org>
* Import the module a lazy `import a.b as c` names
`import a.b as c` compiles to `IMPORT_NAME a.b` followed by `IMPORT_FROM b`.
Lazily, IMPORT_NAME leaves a placeholder holding "a.b", and IMPORT_FROM
rewrote it into the placeholder `lazy from a import b` produces. Reification
then imported `a` alone and read `b` off it, so the module `a.b` was never
imported under its own name: an attribute of the package shadowing it answered
instead, and `math.pi`, which no module backs, bound the float where the eager
statement raises ModuleNotFoundError.
Mark the dotted import on the placeholder and keep the whole name on it.
Reification imports that name and then walks its components with IMPORT_FROM,
which is what the eager statement does.
The test pinning `lazy import math.pi as pi` as working is inverted, since the
eager statement raises.
* Stop excluding test_trace from the lazy-imports-all run
It passes now that a lazy `import a.b as c` imports the module: the
KeyError on 'test.tracedmodules.testmod' came from the submodule never
being imported under its own name.
* Rename lz_submodule to lz_dotted_as and trim the comments
The flag means "bind the whole dotted name, not the root", which the old
name did not say, and import.c already has unrelated lazy_pending_submodules
machinery to be confused with.
* Chain lazy IMPORT_FROM placeholders instead of flagging dotted imports
Each deferred IMPORT_FROM off a placeholder without a fromlist now keeps
the previous placeholder in lz_from and the attribute name in lz_attr.
Reification walks back to the placeholder IMPORT_NAME left, runs that
import, and replays the lookups in order with _PyEval_ImportFrom, which
is what the eager bytecode does. This drops the lz_dotted_as flag and
also follows a custom __lazy_import__ that returns a placeholder for a
different module name.
* Chain every deferred IMPORT_FROM onto the previous placeholder
`lazy from a import b` now records its lookup the same way as `import
a.b as c`, so a placeholder holds either the module name and fromlist or
the previous placeholder and an attribute name, and reification has a
single path. The import passes only the name being resolved as the
fromlist, so accessing b still does not import the other names'
submodules.
* Treat an empty fromlist on a lazy import placeholder as no fromlist
__import__("a.b", fromlist=()) returns the top-level package `a`, the same
as fromlist=None, but the placeholder kept the empty tuple. Every consumer
of lz_attr then read it as a real fromlist: reification narrowed it to the
chained attribute and replayed the lookups on `a.b`, _PyEval_LazyImportFrom
took the attribute off sys.modules["a.b"], and the repr named `a.b.attr`.
_PyLazyImport_New already collapses None to NULL for exactly this reason,
so collapse an empty tuple there too and every site follows.
* gh-157757: Preserve empty fromlists for custom import hooks
---------
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* Fix not raising on non-module import
* Fix traceback tests
* Fix doc string and add new test
* Fix up feedback on tests
* Add test case for lazy import dotted.name as name
In free-threading builds, concurrent calls to sys.getdlopenflags() and
sys.setdlopenflags() race on interp->imports.dlopenflags. Fix by
using FT_ATOMIC_LOAD_INT_RELAXED / FT_ATOMIC_STORE_INT_RELAXED in
_PyImport_GetDLOpenFlags and _PyImport_SetDLOpenFlags, consistent with
how analogous interpreter-state integer fields (lazy_imports_mode,
pystats_enabled) are protected.
Relaxed ordering is correct here: dlopenflags is a standalone config
integer with no ordering relationship to other memory.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Rename from _Py_INTERNAL_ABI_SLOT to _Py_ABI_SLOT
and define the macro using _PyABIInfo_DEFAULT.
Use the ABI slot in stdlib extension modules to enable running
a check of ABI version compatibility.
_tkinter, _tracemalloc and readline don't use the slots, hence they need
explicit handling.
Co-authored-by: Victor Stinner <vstinner@python.org>
Fix a race condition where a thread could receive a partially-initialized
module when another thread's import fails. The race occurs when:
1. Thread 1 starts importing, adds module to sys.modules
2. Thread 2 sees the module in sys.modules via the fast path
3. Thread 1's import fails, removes module from sys.modules
4. Thread 2 returns a stale module reference not in sys.modules
The fix adds verification after the "skip lock" optimization in both Python
and C code paths to check if the module is still in sys.modules. If the
module was removed (due to import failure), we retry the import so the
caller receives the actual exception from the import failure rather than
a stale module reference.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This needs a single bit, but was stored as a void* in the module
struct. This didn't matter due to packing, but now that there's
another bool in the struct, we can save a bit of memory by
making md_gil a bool.
Variables that changed type are renamed, to detect conflicts.
Python has required thread local support since 3.12 (see GH-103324). By assuming that thread locals are always supported, we can improve the performance of third-party extensions by allowing them to access the attached thread and interpreter states directly.
For several builtin functions, we now fall back to __main__.__dict__ for the globals
when there is no current frame and _PyInterpreterState_IsRunningMain() returns
true. This allows those functions to be run with Interpreter.call().
The affected builtins:
* exec()
* eval()
* globals()
* locals()
* vars()
* dir()
We take a similar approach with "stateless" functions, which don't use any
global variables.
