PEP 698 – Override Decorator for Static Typing | peps.python.org
PEP 698 – Override Decorator for Static Typing PEP 698 – Override Decorator for Static Typing Author : Steven Troxler <steven.troxler at gmail.com>, Joshua Xu <jxu425 at fb.com>, Shannon Zhu <szhu at fb.com> Sponsor : Jelle Zijlstra <jelle.zijlstra at gmail.com> Discussions-To : Discourse thread Status : Final Type : Standards Track Topic : Typing Created : 05-Sep-2022 Python-Version : 3.12 Post-History : 20-May-2022 , 17-Aug-2022 , 11-Oct-2022 , 07-Nov-2022 Resolution : Discourse message Table of Contents Abstract Motivation Safe Refactoring Rationale Subclass Implementations Become More Explicit Precedent in Other Languages and Runtime Libraries Static Override Checks in Other Languages Runtime Override Checks in Python Disadvantages Specification No New Rules for Override Compatibility Strict Enforcement Per-Project Motivation Precedent Backward Compatibility Runtime Behavior Set __override__ = True when possible Limitations of setting __override__ Rejected Alternatives Rely on Integrated Development Environments for safety Runtime enforcement Mark a base class to force explicit overrides on subclasses Include the name of the ancestor class being overridden Reference Implementation Copyright Important This PEP is a historical document: see @override and @typing.override for up-to-date specs and documentation.
Canonical typing specs are maintained at the typing specs site ; runtime typing behaviour is described in the CPython documentation. × See the typing specification update process for how to propose changes to the typing spec. Abstract This PEP proposes adding an @override decorator to the Python type system. This will allow type checkers to prevent a class of bugs that occur when a base class changes methods that are inherited by derived classes.
Motivation A primary purpose of type checkers is to flag when refactors or changes break pre-existing semantic structures in the code, so users can identify and make fixes across their project without doing a manual audit of their code. Safe Refactoring Python’s type system does not provide a way to identify call sites that need to be changed to stay consistent when an overridden function API changes. This makes refactoring and transforming code more dangerous.
Consider this simple inheritance structure: class Parent : def foo ( self , x : int ) -> int : return x class Child ( Parent ): def foo ( self , x : int ) -> int : return x + 1 def parent_callsite ( parent : Parent ) -> None : parent . foo ( 1 ) def child_callsite ( child : Child ) -> None : child . foo ( 1 ) If the overridden method on the superclass is renamed or deleted, type checkers will only alert us to update call sites that deal with the base type directly.
But the type checker can only see the new code, not the change we made, so it has no way of knowing that we probably also needed to rename the same method on child classes. A type checker will happily accept this code, even though we are likely introducing bugs: class Parent : # Rename this method def new_foo ( self , x : int ) -> int : return x class Child ( Parent ): # This (unchanged) method used to override `foo` but is unrelated to `new_foo` def foo ( self , x : int ) -> int : return x + 1 def parent_callsite ( parent : Parent ) -> None : # If we pass a Child instance we’ll now run Parent.new_foo - likely a bug parent . new_foo ( 1 ) def child_callsite ( child : Child ) -> None : # We probably wanted to invoke new_foo here.
Instead, we forked the method child . foo ( 1 ) This code will type check, but there are two potential sources of bugs: If we pass a Child instance to the parent_callsite function, it will invoke the implementation in Parent.new_foo . rather than Child.foo . This is probably a bug - we presumably would not have written Child.foo in the first place if we didn’t need custom behavior. Our system was likely relying on Child.foo behaving in a similar way to Parent.foo .
But unless we catch this early, we have now forked the methods, and in future refactors it is likely no one will realize that major changes to the behavior of new_foo likely require updating Child.foo as well, which could lead to major bugs later. The incorrectly-refactored code is type-safe, but is probably not what we intended and could cause our system to behave incorrectly. The bug can be difficult to track down because our new code likely does execute without throwing exceptions.
Tests are less likely to catch the problem, and silent errors can take longer to track down in production. We are aware of several production outages in multiple typed codebases caused by such incorrect refactors. This is our primary motivation for adding an @override decorator to the type system, which lets developers express the relationship between Parent.foo and Child.foo so that type checkers can detect the problem.
Rationale Subclass Implementations Become More Explicit We believe that explicit overrides will make unfamiliar code easier to read than implicit overrides. A developer reading the implementation of a subclass that uses @override can immediately see which methods are overriding functionality in some base class; without this decorator, the only way to quickly find out is using a static analysis tool. Precedent in Other Languages and Runtime Libraries Static Override Checks in Other Languages Many popular programming languages support override checks.
For example: C++ has override . C# has override . Hack has <<__Override>> . Java has @Override . Kotlin has override . Scala has override . Swift has override . TypeScript has override . Runtime Override Checks in Python Today, there is an Overrides library that provides decorators @overrides [sic] and @final and will enforce them at runtime. PEP 591 added a @final decorator with the same semantics as those in the Overrides library.
But the override component of the runtime library is not supported statically at all, which has added some confusion around the mix/matched support. Providing support for @override in static checks would add value because: Bugs can be caught earlier, often in-editor. Static checks come with no performance overhead, unlike runtime checks. Bugs will be caught quickly even in rarely-used modules, whereas with runtime checks these might go undetected for a time without automated tests of all imports.
Disadvantages Using @override will make code more verbose. Specification When type checkers encounter a method decorated with @typing.override they should treat it as a type error unless that method is overriding a compatible method or attribute in some ancestor class. from typing import override class Parent : def foo ( self ) -> int : return 1 def bar ( self , x : str ) -> str : return x class Child ( Parent ): @override def foo ( self ) -> int : return 2 @override def baz ( self ) -> int : # Type check error: no matching signature in ancestor return 1 The @override decorator should be permitted anywhere a type checker considers a method to be a valid override, which typically includes not only normal methods but also @property , @staticmethod , and @classmethod .
No New Rules for Override Compatibility This PEP is exclusively concerned with the handling of the new @override decorator, which specifies that the decorated method must override some attribute in an ancestor class. This PEP does not propose any new rules regarding the type signatures of such methods. Strict Enforcement Per-Project We believe that @override is most useful if checkers also allow developers to opt into a strict mode where methods that override a parent class are required to use the decorator.
Strict enforcement should be opt-in for backward compatibility. Motivation The primary reason for a strict mode that requires @override is that developers can only trust that refactors are override-safe if they know that the @override decorator is used throughout the project. There is another class of bug related to overrides that we can only catch using a
The rest of this page is on the live site.