Text of peps.python.org › pep-0302 as Styx crawled it on Nov 20, 2026. The page may have changed since. Only its words are kept, without pictures or layout, and only the first part of a long page.

Open the live page

PEP 302 – New Import Hooks | peps.python.org

PEP 302 – New Import Hooks PEP 302 – New Import Hooks Author : Just van Rossum <just at letterror.com>, Paul Moore <p.f.moore at gmail.com> Status : Final Type : Standards Track Created : 19-Dec-2002 Python-Version : 2.3 Post-History : 19-Dec-2002 Table of Contents Abstract Motivation Use cases Rationale Specification part 1: The Importer Protocol Specification part 2: Registering Hooks Packages and the role of __path__ Optional Extensions to the Importer Protocol Integration with the ‘imp’ module Forward Compatibility Open Issues Implementation References and Footnotes Copyright Warning The language reference for import [10] and importlib documentation [11] now supersede this PEP.

This document is no longer updated and provided for historical purposes only. Abstract This PEP proposes to add a new set of import hooks that offer better customization of the Python import mechanism. Contrary to the current __import__ hook, a new-style hook can be injected into the existing scheme, allowing for a finer grained control of how modules are found and how they are loaded. Motivation The only way to customize the import mechanism is currently to override the built-in __import__ function.

However, overriding __import__ has many problems. To begin with: An __import__ replacement needs to fully reimplement the entire import mechanism, or call the original __import__ before or after the custom code. It has very complex semantics and responsibilities. __import__ gets called even for modules that are already in sys.modules , which is almost never what you want, unless you’re writing some sort of monitoring tool.

The situation gets worse when you need to extend the import mechanism from C: it’s currently impossible, apart from hacking Python’s import.c or reimplementing much of import.c from scratch. There is a fairly long history of tools written in Python that allow extending the import mechanism in various way, based on the __import__ hook. The Standard Library includes two such tools: ihooks.py (by GvR) and imputil.py [1] (Greg Stein), but perhaps the most famous is iu.py by Gordon McMillan, available as part of his Installer package.

Their usefulness is somewhat limited because they are written in Python; bootstrapping issues need to worked around as you can’t load the module containing the hook with the hook itself. So if you want the entire Standard Library to be loadable from an import hook, the hook must be written in C. Use cases This section lists several existing applications that depend on import hooks. Among these, a lot of duplicate work was done that could have been saved if there had been a more flexible import hook at the time.

This PEP should make life a lot easier for similar projects in the future. Extending the import mechanism is needed when you want to load modules that are stored in a non-standard way. Examples include modules that are bundled together in an archive; byte code that is not stored in a pyc formatted file; modules that are loaded from a database over a network. The work on this PEP was partly triggered by the implementation of PEP 273 , which adds imports from Zip archives as a built-in feature to Python.

While the PEP itself was widely accepted as a must-have feature, the implementation left a few things to desire. For one thing it went through great lengths to integrate itself with import.c , adding lots of code that was either specific for Zip file imports or not specific to Zip imports, yet was not generally useful (or even desirable) either. Yet the PEP 273 implementation can hardly be blamed for this: it is simply extremely hard to do, given the current state of import.c .

Packaging applications for end users is a typical use case for import hooks, if not the typical use case. Distributing lots of source or pyc files around is not always appropriate (let alone a separate Python installation), so there is a frequent desire to package all needed modules in a single file. So frequent in fact that multiple solutions have been implemented over the years. The oldest one is included with the Python source code: Freeze [2] .

It puts marshalled byte code into static objects in C source code. Freeze’s “import hook” is hard wired into import.c , and has a couple of issues. Later solutions include Fredrik Lundh’s Squeeze, Gordon McMillan’s Installer, and Thomas Heller’s py2exe [3] . MacPython ships with a tool called BuildApplication . Squeeze, Installer and py2exe use an __import__ based scheme (py2exe currently uses Installer’s iu.py , Squeeze used ihooks.py ), MacPython has two Mac-specific import hooks hard wired into import.c , that are similar to the Freeze hook.

The hooks proposed in this PEP enables us (at least in theory; it’s not a short-term goal) to get rid of the hard coded hooks in import.c , and would allow the __import__ -based tools to get rid of most of their import.c emulation code. Before work on the design and implementation of this PEP was started, a new BuildApplication -like tool for Mac OS X prompted one of the authors of this PEP (JvR) to expose the table of frozen modules to Python, in the imp module.

The main reason was to be able to use the freeze import hook (avoiding fancy __import__ support), yet to also be able to supply a set of modules at runtime. This resulted in issue #642578 [4] , which was mysteriously accepted (mostly because nobody seemed to care either way ;-). Yet it is completely superfluous when this PEP gets accepted, as it offers a much nicer and general way to do the same thing. Rationale While experimenting with alternative implementation ideas to get built-in Zip import, it was discovered that achieving this is possible with only a fairly small amount of changes to import.c .

This allowed to factor out the Zip-specific stuff into a new source file, while at the same time creating a general new import hook scheme: the one you’re reading about now. An earlier design allowed non-string objects on sys.path . Such an object would have the necessary methods to handle an import. This has two disadvantages: 1) it breaks code that assumes all items on sys.path are strings; 2) it is not compatible with the PYTHONPATH environment variable.

The latter is directly needed for Zip imports. A compromise came from Jython: allow string subclasses on sys.path , which would then act as importer objects. This avoids some breakage, and seems to work well for Jython (where it is used to load modules from .jar files), but it was perceived as an “ugly hack”. This led to a more elaborate scheme, (mostly copied from McMillan’s iu.py ) in which each in a list of candidates is asked whether it can handle the sys.path item, until one is found that can.

This list of candidates is a new object in the sys module: sys.path_hooks . Traversing sys.path_hooks for each path item for each new import can be expensive, so the results are cached in another new object in the sys module: sys.path_importer_cache . It maps sys.path entries to importer objects. To minimize the impact on import.c as well as to avoid adding extra overhead, it was chosen to not add an explicit hook and importer object for the existing file system import logic (as iu.py has), but to simply fall back to the built-in logic if no hook on sys.path_hooks could handle the path item.

If this is the case, a None value is stored in sys.path_importer_cache , again to avoid repeated lookups. (Later we can go further and add a real importer object for the built-in mechanism, for now, the None fallback scheme should suffice.) A question was raised: what about importers that don’t need any entry on sys.path ? (Built-in and frozen modules fall into that category.) Again, Gordon McMillan to the rescue: iu.py contains a thing he calls the metapath .

In this PEP’s implementation, it’s a list of importer objects that is traversed before sys.path . This list is yet another new object in the sys module: sys.meta_path . Curren

The rest of this page is on the live site.