Text of peps.python.org › pep-0744 as Styx crawled it on Nov 18, 2026. The page may have changed since. Only its words are kept, without pictures or layout, and only the first part of a long page.

Open the live page

PEP 744 – JIT Compilation | peps.python.org

PEP 744 – JIT Compilation PEP 744 – JIT Compilation Author : Brandt Bucher <brandt at python.org>, Savannah Ostrowski <savannah at python.org> Discussions-To : Discourse thread Status : Draft Type : Informational Created : 11-Apr-2024 Python-Version : 3.13 Post-History : 11-Apr-2024 Table of Contents Abstract Motivation Rationale Specification Support Backwards Compatibility Debugging Security Implications Apple Silicon How to Teach This Reference Implementation Rejected Ideas Maintain it outside of CPython Turn it on by default Support multiple compiler toolchains Compile the base interpreter’s bytecode Add GPU support Open Issues Speed Memory Dependencies Footnotes Copyright Abstract Earlier this year, an experimental “just-in-time” compiler was merged into CPython’s main development branch.

While recent CPython releases have included other substantial internal changes, this addition represents a particularly significant departure from the way CPython has traditionally executed Python code. As such, it deserves wider discussion. This PEP aims to summarize the design decisions behind this addition, the current state of the implementation, and future plans for making the JIT a permanent, non-experimental part of CPython.

It does not seek to provide a comprehensive overview of how the JIT works, instead focusing on the particular advantages and disadvantages of the chosen approach, as well as answering many questions that have been asked about the JIT since its introduction. Readers interested in learning more about the new JIT are encouraged to consult the following resources: The presentation which first introduced the JIT at the 2023 CPython Core Developer Sprint.

It includes relevant background, a light technical introduction to the “copy-and-patch” technique used, and an open discussion of its design amongst the core developers present. Slides for this talk can be found on GitHub . The open access paper originally describing copy-and-patch. The blog post by the paper’s author detailing the implementation of a copy-and-patch JIT compiler for Lua. While this is a great low-level explanation of the approach, note that it also incorporates other techniques and makes implementation decisions that are not particularly relevant to CPython’s JIT.

The implementation itself. Motivation Until this point, CPython has always executed Python code by compiling it to bytecode, which is interpreted at runtime. This bytecode is a more-or-less direct translation of the source code: it is untyped, and largely unoptimized. Since the Python 3.11 release, CPython has used a “specializing adaptive interpreter” ( PEP 659 ), which rewrites these bytecode instructions in-place with type-specialized versions as they run.

This new interpreter delivers significant performance improvements, despite the fact that its optimization potential is limited by the boundaries of individual bytecode instructions. It also collects a wealth of new profiling information: the types flowing though a program, the memory layout of particular objects, and what paths through the program are being executed the most. In other words, what to optimize, and how to optimize it.

Since the Python 3.12 release, CPython has generated this interpreter from a C-like domain-specific language (DSL). In addition to taming some of the complexity of the new adaptive interpreter, the DSL also allows CPython’s maintainers to avoid hand-writing tedious boilerplate code in many parts of the interpreter, compiler, and standard library that must be kept in sync with the instruction definitions. This ability to generate large amounts of runtime infrastructure from a single source of truth is not only convenient for maintenance; it also unlocks many possibilities for expanding CPython’s execution in new ways.

For instance, it makes it feasible to automatically generate tables for translating a sequence of instructions into an equivalent sequence of smaller “micro-ops”, generate an optimizer for sequences of these micro-ops, and even generate an entire second interpreter for executing them. In fact, since early in the Python 3.13 release cycle, all CPython builds have included this exact micro-op translation, optimization, and execution machinery.

However, it is disabled by default; the overhead of interpreting even optimized traces of micro-ops is just too large for most code. Heavier optimization probably won’t improve the situation much either, since any efficiency gains made by new optimizations will likely be offset by the interpretive overhead of even smaller, more complex micro-ops. The most obvious strategy to overcome this new bottleneck is to statically compile these optimized traces.

This presents opportunities to avoid several sources of indirection and overhead introduced by interpretation. In particular, it allows the removal of dispatch overhead between micro-ops (by replacing a generic interpreter with a straight-line sequence of hot code), instruction decoding overhead for individual micro-ops (by “burning” the values or addresses of arguments, constants, and cached values directly into machine instructions), and memory traffic (by moving data off of heap-allocated Python frames and into physical hardware registers).

Since much of this data varies even between identical runs of a program and the existing optimization pipeline makes heavy use of runtime profiling information, it doesn’t make much sense to compile these traces ahead of time and would be a substantial redesign of the existing specification and micro-op tracing infrastructure that has already been implemented. As has been demonstrated for many other dynamic languages ( and even Python itself ), the most promising approach is to compile the optimized micro-ops “just in time” for execution.

Rationale Despite their reputation, JIT compilers are not magic “go faster” machines. Developing and maintaining any sort of optimizing compiler for even a single platform, let alone all of CPython’s most popular supported platforms, is an incredibly complicated, expensive task. Using an existing compiler framework like LLVM can make this task simpler, but only at the cost of introducing heavy runtime dependencies and significantly higher JIT compilation overhead.

It’s clear that successfully compiling Python code at runtime requires not only high-quality Python-specific optimizations for the code being run, but also quick generation of efficient machine code for the optimized program. The Python core development team has the necessary skills and experience for the former (a middle-end tightly coupled to the interpreter), and copy-and-patch compilation provides an attractive solution for the latter.

In a nutshell, copy-and-patch allows a high-quality template JIT compiler to be generated from the same DSL used to generate the rest of the interpreter. For a widely-used, volunteer-driven project like CPython, this benefit cannot be overstated: CPython’s maintainers, by merely editing the bytecode definitions, will also get the JIT backend updated “for free”, for all JIT-supported platforms, at once. This is equally true whether instructions are being added, modified, or removed.

Like the rest of the interpreter, the JIT compiler is generated at build time, and has no runtime dependencies. It supports a wide range of platforms (see the Support section below), and has comparatively low maintenance burden. In all, the current implementation is made up of about 900 lines of build-time Python code and 500 lines of runtime C code. Specification The JIT is currently not part of the default build configuration, and it is likely to remain that way for the foreseeable future (though official binaries may include it).

That said, the JIT will become non-experimental once all of the following conditions are met: It provides a meaningful performance improvement for at least one popular platform (realistically, on

The rest of this page is on the live site.