PEP 487 – Simpler customisation of class creation | peps.python.org
PEP 487 – Simpler customisation of class creation PEP 487 – Simpler customisation of class creation Author : Martin Teichmann <lkb.teichmann at gmail.com> Status : Final Type : Standards Track Created : 27-Feb-2015 Python-Version : 3.6 Post-History : 27-Feb-2015, 05-Feb-2016, 24-Jun-2016, 02-Jul-2016, 13-Jul-2016 Replaces : 422 Resolution : Python-Dev message Table of Contents Abstract Background Proposal Key Benefits Easier inheritance of definition time behaviour Reduced chance of metaclass conflicts New Ways of Using Classes Subclass registration Trait descriptors Implementation Details Reference Implementation Backward compatibility issues Rejected Design Options Calling the hook on the class itself Other variants of calling the hooks Requiring an explicit decorator on __init_subclass__ A more __new__ -like hook Adding a class attribute with the attribute order History Copyright Abstract Currently, customising class creation requires the use of a custom metaclass.
This custom metaclass then persists for the entire lifecycle of the class, creating the potential for spurious metaclass conflicts. This PEP proposes to instead support a wide range of customisation scenarios through a new __init_subclass__ hook in the class body, and a hook to initialize attributes. The new mechanism should be easier to understand and use than implementing a custom metaclass, and thus should provide a gentler introduction to the full power of Python’s metaclass machinery.
Background Metaclasses are a powerful tool to customize class creation. They have, however, the problem that there is no automatic way to combine metaclasses. If one wants to use two metaclasses for a class, a new metaclass combining those two needs to be created, typically manually. This need often occurs as a surprise to a user: inheriting from two base classes coming from two different libraries suddenly raises the necessity to manually create a combined metaclass, where typically one is not interested in those details about the libraries at all.
This becomes even worse if one library starts to make use of a metaclass which it has not done before. While the library itself continues to work perfectly, suddenly every code combining those classes with classes from another library fails. Proposal While there are many possible ways to use a metaclass, the vast majority of use cases falls into just three categories: some initialization code running after class creation, the initialization of descriptors and keeping the order in which class attributes were defined.
The first two categories can easily be achieved by having simple hooks into the class creation: An __init_subclass__ hook that initializes all subclasses of a given class. upon class creation, a __set_name__ hook is called on all the attribute (descriptors) defined in the class, and The third category is the topic of another PEP, PEP 520 . As an example, the first use case looks as follows: >>> class QuestBase : ... # this is implicitly a @classmethod (see below for motivation) ... def __init_subclass__ ( cls , swallow , ** kwargs ): ... cls . swallow = swallow ... super () . __init_subclass__ ( ** kwargs ) >>> class Quest ( QuestBase , swallow = "african" ): ... pass >>> Quest . swallow 'african' The base class object contains an empty __init_subclass__ method which serves as an endpoint for cooperative multiple inheritance.
Note that this method has no keyword arguments, meaning that all methods which are more specialized have to process all keyword arguments. This general proposal is not a new idea (it was first suggested for inclusion in the language definition more than 10 years ago , and a similar mechanism has long been supported by Zope’s ExtensionClass ), but the situation has changed sufficiently in recent years that the idea is worth reconsidering for inclusion.
The second part of the proposal adds an __set_name__ initializer for class attributes, especially if they are descriptors. Descriptors are defined in the body of a class, but they do not know anything about that class, they do not even know the name they are accessed with. They do get to know their owner once __get__ is called, but still they do not know their name. This is unfortunate, for example they cannot put their associated value into their object’s __dict__ under their name, since they do not know that name.
This problem has been solved many times, and is one of the most important reasons to have a metaclass in a library. While it would be easy to implement such a mechanism using the first part of the proposal, it makes sense to have one solution for this problem for everyone. To give an example of its usage, imagine a descriptor representing weak referenced values: import weakref class WeakAttribute : def __get__ ( self , instance , owner ): return instance . __dict__ [ self . name ]() def __set__ ( self , instance , value ): instance . __dict__ [ self . name ] = weakref . ref ( value ) # this is the new initializer: def __set_name__ ( self , owner , name ): self . name = name Such a WeakAttribute may, for example, be used in a tree structure where one wants to avoid cyclic references via the parent: class TreeNode : parent = WeakAttribute () def __init__ ( self , parent ): self . parent = parent Note that the parent attribute is used like a normal attribute, yet the tree contains no cyclic references and can thus be easily garbage collected when out of use.
The parent attribute magically becomes None once the parent ceases existing. While this example looks very trivial, it should be noted that until now such an attribute cannot be defined without the use of a metaclass. And given that such a metaclass can make life very hard, this kind of attribute does not exist yet. Initializing descriptors could simply be done in the __init_subclass__ hook. But this would mean that descriptors can only be used in classes that have the proper hook, the generic version like in the example would not work generally.
One could also call __set_name__ from within the base implementation of object.__init_subclass__ . But given that it is a common mistake to forget to call super() , it would happen too often that suddenly descriptors are not initialized. Key Benefits Easier inheritance of definition time behaviour Understanding Python’s metaclasses requires a deep understanding of the type system and the class construction process. This is legitimately seen as challenging, due to the need to keep multiple moving parts (the code, the metaclass hint, the actual metaclass, the class object, instances of the class object) clearly distinct in your mind.
Even when you know the rules, it’s still easy to make a mistake if you’re not being extremely careful. Understanding the proposed implicit class initialization hook only requires ordinary method inheritance, which isn’t quite as daunting a task. The new hook provides a more gradual path towards understanding all of the phases involved in the class definition process. Reduced chance of metaclass conflicts One of the big issues that makes library authors reluctant to use metaclasses (even when they would be appropriate) is the risk of metaclass conflicts.
These occur whenever two unrelated metaclasses are used by the desired parents of a class definition. This risk also makes it very difficult to add a metaclass to a class that has previously been published without one. By contrast, adding an __init_subclass__ method to an existing type poses a similar level of risk to adding an __init__ method: technically, there is a risk of breaking poorly implemented subclasses, but when that occurs, it is recognised as a bug in the subclass rather than the library author breaching backwards compatibility guarantees.
New Ways of Using Classes Subclass registration Especially when writing a plugin system, one likes to register new subclasses of a plugin baseclass. This can be done as follows: class PluginBase : subclasses = [] def __init_s
The rest of this page is on the live site.