From 4679de7ca8abfe842613f28788e47358aa977843 Mon Sep 17 00:00:00 2001 From: Dmitry Voropaev Date: Fri, 25 Sep 2026 00:17:39 +0300 Subject: [PATCH] gh-77391: Clarify what pkgutil.walk_packages() does with prefix The docs describe prefix as "a string to output on the front of every module name on output". That is accurate for iter_modules(), but it leaves out the part that catches people in walk_packages(): the same string goes into the name each package is imported under while the walk descends, so the descent only reaches a package when prefix + name resolves to the very package that was found on path. When it resolves to something else, that other package is walked instead. When importing it raises ImportError, the submodules are absent and nothing in the results says so. Walking ctypes.__path__ with no prefix on the current main lists six names and quietly loses the three modules under ctypes.macholib; with ctypes.__name__ + '.' it lists all nine. Passing onerror does not make up for it, which is easy to assume from the old wording: it reports macholib, and the output is still the same six names. Only the wording changes, in the rst and in the docstring that repeats it. The implementation problem Nick Coghlan described on the same report, that the descent goes through a name-based __import__ instead of loading the package from the file it has already found, is left alone here; pull request #11956 is already open for it. --- Doc/library/pkgutil.rst | 9 ++++++++- Lib/pkgutil.py | 11 +++++++++-- .../2026-09-25-00-15-48.gh-issue-77391.1utlEd.rst | 3 +++ 3 files changed, 20 insertions(+), 3 deletions(-) create mode 100644 Misc/NEWS.d/next/Documentation/2026-09-25-00-15-48.gh-issue-77391.1utlEd.rst diff --git a/Doc/library/pkgutil.rst b/Doc/library/pkgutil.rst index 5473a367c49a3a..54455d8ab77090 100644 --- a/Doc/library/pkgutil.rst +++ b/Doc/library/pkgutil.rst @@ -112,10 +112,17 @@ support. *path* should be either ``None`` or a list of paths to look for modules in. *prefix* is a string to output on the front of every module name on output. + It also becomes part of the name under which each package is imported + during the recursive descent. Note that this function must import all *packages* (*not* all modules!) on the given *path*, in order to access the ``__path__`` attribute to find - submodules. + submodules. The import is done by name, so ``prefix + name`` must resolve + to the package that was found on *path*. If it resolves to a different + package, the submodules of that one are listed instead. If it raises + :exc:`ImportError`, the submodules are left out and nothing in the results + shows that they are missing. Passing a package's ``__path__`` together + with its ``__name__`` and a trailing dot avoids both. *onerror* is a function which gets called with one argument (the name of the package which was being imported) if any exception occurs while trying to diff --git a/Lib/pkgutil.py b/Lib/pkgutil.py index 9121d6a1e2285c..2332d65b937655 100644 --- a/Lib/pkgutil.py +++ b/Lib/pkgutil.py @@ -45,11 +45,18 @@ def walk_packages(path=None, prefix='', onerror=None): modules in. 'prefix' is a string to output on the front of every module name - on output. + on output. It also becomes part of the name under which each + package is imported during the recursive descent. Note that this function must import all *packages* (NOT all modules!) on the given path, in order to access the __path__ - attribute to find submodules. + attribute to find submodules. The import is done by name, so + prefix + name must resolve to the package that was found on path. + If it resolves to a different package, the submodules of that one + are listed instead. If it raises ImportError, the submodules are + left out and nothing in the results shows that they are missing. + Passing a package's __path__ together with its __name__ and a + trailing dot avoids both. 'onerror' is a function which gets called with one argument (the name of the package which was being imported) if any exception diff --git a/Misc/NEWS.d/next/Documentation/2026-09-25-00-15-48.gh-issue-77391.1utlEd.rst b/Misc/NEWS.d/next/Documentation/2026-09-25-00-15-48.gh-issue-77391.1utlEd.rst new file mode 100644 index 00000000000000..b7a5eda5cae241 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2026-09-25-00-15-48.gh-issue-77391.1utlEd.rst @@ -0,0 +1,3 @@ +Document that the *prefix* argument of :func:`pkgutil.walk_packages` affects +the imports made during the recursive descent, not just the names that are +returned.