PEP 558 – Defined semantics for locals() | peps.python.org
PEP 558 – Defined semantics for locals() PEP 558 – Defined semantics for locals() Author : Alyssa Coghlan <ncoghlan at gmail.com> BDFL-Delegate : Nathaniel J. Smith Discussions-To : Python-Dev list Status : Withdrawn Type : Standards Track Created : 08-Sep-2017 Python-Version : 3.13 Post-History : 08-Sep-2017, 22-May-2019, 30-May-2019, 30-Dec-2019, 18-Jul-2021, 26-Aug-2021 Table of Contents PEP Withdrawal Abstract Motivation Proposal New locals() documentation Module scope Class scope Function scope CPython Implementation Changes Summary of proposed implementation-specific changes Providing the updated Python level semantics Resolving the issues with tracing mode behaviour Fast locals proxy implementation details Changes to the stable C API/ABI Changes to the public CPython C API Reducing the runtime overhead of trace hooks Rationale and Design Discussion Changing locals() to return independent snapshots at function scope Keeping locals() as a snapshot at function scope What happens with the default args for eval() and exec() ?
Additional considerations for eval() and exec() in optimized scopes Retaining the internal frame value cache Changing the frame API semantics in regular operation Continuing to support storing additional data on optimised frames Historical semantics at function scope Proposing several additions to the stable C API/ABI Comparison with PEP 667 Implementation Acknowledgements References Copyright PEP Withdrawal In December 2021, this PEP and PEP 667 converged on a common definition of the proposed changes to the Python level semantics of the locals() builtin (as documented in the PEP text below), with the only remaining differences being in the proposed C API changes and various internal implementation details.
Of those remaining differences, the most significant one was that PEP 667 at the time still proposed an immediate backwards compatibility break for the PyEval_GetLocals() API as soon as the PEP was accepted and implemented. PEP 667 has since been changed to propose a generous deprecation period for the PyEval_GetLocals() API, continuing to support it in parallel with the improved semantics offered by the new PyEval_GetFrameLocals() API.
Any remaining C API design concerns relate to new informational APIs that can be added at a later date if they are deemed necessary, and any potential concerns about the exact performance characteristics of the frame locals view implementation are outweighed by the availability of a viable reference implementation. Accordingly, this PEP has been withdrawn in favour of proceeding with PEP 667 . Note: while implementing PEP 667 it became apparent that the rationale for and impact of locals() being updated to return independent snapshots in optimized scopes was not entirely clear in either PEP.
The Motivation and Rationale sections in this PEP have been updated accordingly (since those aspects are equally applicable to the accepted PEP 667 ). Abstract The semantics of the locals() builtin have historically been underspecified and hence implementation dependent. This PEP proposes formally standardising on the behaviour of the CPython 3.10 reference implementation for most execution scopes, with some adjustments to the behaviour at function scope to make it more predictable and independent of the presence or absence of tracing functions.
In addition, it proposes that the following functions be added to the stable Python C API/ABI: typedef enum { PyLocals_UNDEFINED = -1 , PyLocals_DIRECT_REFERENCE = 0 , PyLocals_SHALLOW_COPY = 1 , _PyLocals_ENSURE_32BIT_ENUM = 2147483647 } PyLocals_Kind ; PyLocals_Kind PyLocals_GetKind (); PyObject * PyLocals_Get (); PyObject * PyLocals_GetCopy (); It also proposes the addition of several supporting functions and type definitions to the CPython C API.
Motivation While the precise semantics of the locals() builtin are nominally undefined, in practice, many Python programs depend on it behaving exactly as it behaves in CPython (at least when no tracing functions are installed). Other implementations such as PyPy are currently replicating that behaviour, up to and including replication of local variable mutation bugs that can arise when a trace hook is installed [1] .
While this PEP considers CPython’s current behaviour when no trace hooks are installed to be largely acceptable, it considers the current behaviour when trace hooks are installed to be problematic, as it causes bugs like [1] without even reliably enabling the desired functionality of allowing debuggers like pdb to mutate local variables [3] . Review of the initial PEP and the draft implementation then identified an opportunity for simplification of both the documentation and implementation of the function level locals() behaviour by updating it to return an independent snapshot of the function locals and closure variables on each call, rather than continuing to return the semi-dynamic intermittently updated shared copy that it has historically returned in CPython.
Specifically, the proposal in this PEP eliminates the historical behaviour where adding a new local variable can change the behaviour of code executed with exec() in function scopes, even if that code runs before the local variable is defined. For example: def f (): exec ( "x = 1" ) print ( locals () . get ( "x" )) f () prints 1 , but: def f (): exec ( "x = 1" ) print ( locals () . get ( "x" )) x = 0 f () prints None (the default value from the .get() call).
With this PEP both examples would print None , as the call to exec() and the subsequent call to locals() would use independent dictionary snapshots of the local variables rather than using the same shared dictionary cached on the frame object. Proposal The expected semantics of the locals() builtin change based on the current execution scope. For this purpose, the defined scopes of execution are: module scope: top-level module code, as well as any other code executed using exec() or eval() with a single namespace class scope: code in the body of a class statement, as well as any other code executed using exec() or eval() with separate local and global namespaces function scope: code in the body of a def or async def statement, or any other construct that creates an optimized code block in CPython (e.g. comprehensions, lambda functions) This PEP proposes elevating most of the current behaviour of the CPython reference implementation to become part of the language specification, except that each call to locals() at function scope will create a new dictionary object, rather than caching a common dict instance in the frame object that each invocation will update and return.
This PEP also proposes to largely eliminate the concept of a separate “tracing” mode from the CPython reference implementation. In releases up to and including Python 3.10, the CPython interpreter behaves differently when a trace hook has been registered in one or more threads via an implementation dependent mechanism like sys.settrace ( [4] ) in CPython’s sys module or PyEval_SetTrace ( [5] ) in CPython’s C API.
If this PEP is accepted, then the only remaining behavioural difference when a trace hook is installed is that some optimisations in the interpreter eval loop are disabled when the tracing logic needs to run after each opcode. This PEP proposes changes to CPython’s behaviour at function scope that make the locals() builtin semantics when a trace hook is registered identical to those used when no trace hook is registered, while also making the related frame API semantics clearer and easier for interactive debuggers to rely on.
The proposed elimination of tracing mode affects the semantics of frame object references obtained through other means, such as via a traceback, or via the sys._getframe() API, as the write-through semantics needed for trace hook support are always provided by the f_locals attribute on frame objects, rather than being runtime state dependent. New locals() documentation The h
The rest of this page is on the live site.