gh-155695: Remove resolved names from sys.lazy_modules consistently - #157714
brittanyrey wants to merge 5 commits into
Conversation
|
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated. Once you have made the requested changes, please leave a comment on this pull request containing the phrase |
|
@pablogsal Addressed comments. |
|
cc @encukou |
|
Thanks! The initializing-module case is covered now. I left one follow-up comment about the new |
Names only left sys.lazy_modules through _imp._set_lazy_attributes(), which the import machinery calls from _find_and_load_unlocked(). Two cases never reached it, so their names were recorded and then kept forever: - A lazy import of a module already in sys.modules. _find_and_load() returns early, so nothing ever discards the name. Do not record it in the first place. - The "pkg.attr" entry for `lazy from pkg import attr`. The import machinery only discards module names, and attr is often not a module. Discard it when the lazy object is reified, where the name is already known and has been resolved either way. Submodules that are not yet loaded are still tracked: loading a package does not load its submodules, so those imports can still fire. Names whose reification failed also stay tracked, since the import can still happen.
A module is in sys.modules while its body runs, so lazy_modules_add() counted it as loaded and skipped recording the name. If the body then raises, the module is removed from sys.modules again and the lazy import is left pending under no name at all. Check __spec__._initializing so such a module does not count as loaded. A name added while a module initializes is still discarded once the import completes, since _set_lazy_attributes() runs after the body.
pablogsal
added
the
needs backport to 3.15
374d85f to
44ebbaa
Compare
|
I found a couple of issues and pushed fixes. Checking the spec could call an Importing an already loaded attribute could also leave a stale entry in |
Read stored initialization flags without invoking spec callbacks or materializing dictionaries. Keep unsupported spec state conservatively tracked. Skip cached exports without changing their import context or clearing independently pending modules.
83ac500 to
ad36b56
Compare
2 participants
summary
Improve
sys.lazy_modulesto address the following issues:sys.lazy_modules.lazy from pkg import attris cleaned up after reification.perf
End-to-end (hyperfine,
--warmup 50, 1000 runs; 400 for reify-only)-X lazy_imports=allapp