PEP 751 – A file format to record Python dependencies for installation reproducibility | peps.python.org
PEP 751 – A file format to record Python dependencies for installation reproducibility PEP 751 – A file format to record Python dependencies for installation reproducibility Author : Brett Cannon <brett at python.org> Status : Final Type : Standards Track Topic : Packaging Created : 24-Jul-2024 Post-History : 25-Jul-2024 30-Oct-2024 15-Jan-2025 Replaces : 665 Resolution : 31-Mar-2025 Table of Contents Abstract Motivation Rationale Specification File Name File Format lock-version environments requires-python extras dependency-groups default-groups created-by [[packages]] packages.name packages.version packages.marker packages.requires-python [[packages.dependencies]] [packages.vcs] packages.vcs.type packages.vcs.url packages.vcs.path packages.vcs.requested-revision packages.vcs.commit-id packages.vcs.subdirectory [packages.directory] packages.directory.path packages.directory.editable packages.directory.subdirectory [packages.archive] packages.archive.url packages.archive.path packages.archive.size packages.archive.upload-time [packages.archive.hashes] packages.archive.subdirectory packages.index [packages.sdist] packages.sdist.name packages.sdist.upload-time packages.sdist.url packages.sdist.path packages.sdist.size packages.sdist.hashes [[packages.wheels]] packages.wheels.name packages.wheels.upload-time packages.wheels.url packages.wheels.path packages.wheels.size packages.wheels.hashes [[packages.attestation-identities]] packages.attestation-identities.kind [packages.tool] [tool] Additions to marker expression syntax Example Installation Semantic differences with requirements.txt files Backwards Compatibility Security Implications How to Teach This Reference Implementation Rejected Ideas Recording the dependency graph for installation purposes Specifying a new core metadata version that requires consistent metadata across files Have the installer do dependency resolution Requiring minimum hash algorithm support File naming Using *.pylock.toml as the file name Using *.pylock as the file name Not having a naming convention for the file File format Use JSON over TOML Use YAML over TOML Other keys A single hash algorithm for the whole file Hashing the contents of the lock file itself Recording the creation date of the lock file Recording the package indexes used in searching Locking build requirements for sdists A dedicated direct key Simplification Drop recording the package version Drop the requirement to specify the location of an sdist and/or wheels Drop requiring file size and hashes Drop recording the sdist file name Make packages.wheels a table Self-Referential Drop the [tool] table List the requirement inputs for the file Auditing Recording dependents Acknowledgements Copyright Important This PEP is a historical document.
The up-to-date, canonical spec, pylock.toml Specification , is maintained on the PyPA specs page . × See the PyPA specification update process for how to propose changes. Abstract This PEP proposes a new file format for specifying dependencies to enable reproducible installation in a Python environment. The format is designed to be human-readable and machine-generated. Installers consuming the file should be able to calculate what to install without the need for dependency resolution at install-time.
Motivation Currently, no standard exists to create an immutable record, such as a lock file, which specifies what direct and indirect dependencies should be installed into a virtual environment. Considering there are at least five well-known solutions to this problem in the community ( PDM , pip freeze , pip-tools , Poetry , and uv ), there seems to be an appetite for lock files in general. Those tools also vary in what locking scenarios they support.
For instance, pip freeze and pip-tools only generate single-use lock files for the current environment while PDM, Poetry, and uv can/try to lock for multiple environments and use-cases at once. There’s also concerns around the lack of secure defaults in the face of supply chain attacks by some tools (e.g. including hashes for files). The lack of a standard also has some drawbacks. For instance, any tooling that wants to work with lock files must choose which format to support, potentially leaving users unsupported (e.g.
Dependabot only supporting select tools, same for cloud providers who can do dependency installations on your behalf, etc.). It also impacts portability between tools, which causes vendor lock-in. By not having compatibility and interoperability it fractures tooling around lock files where both users and tools have to choose what lock file format to use upfront, making it costly to use/switch to other formats (e.g. tooling around auditing a lock file).
Rallying around a single format removes this cost/barrier. The closest the community has to a standard are pip’s requirements files which all the aforementioned tools either use directly as their file format or export to (i.e. requirements.txt ). Unfortunately, the format is not a standard but is supported by convention. It’s also designed very much for pip’s needs, limiting its flexibility and ease of use (e.g. it’s a bespoke file format).
Lastly, it is not secure by default (e.g. file hash support is entirely an opt-in feature, you have to tell pip to not look for other dependencies outside of what’s in the requirements file, etc.). Note Much of the motivation from PEP 665 also applies to this PEP. Rationale The file format proposed by this PEP is designed to be human-readable. This is so that the contents of the file can be audited by a human to make sure no undesired dependencies end up being included in the lock file.
The file format is also designed to not require a resolver at install time. This greatly simplifies reasoning about what would be installed when consuming a lock file. It should also lead to faster installs which are much more frequent than creating a lock file. The data in the file should be consumable by tools not written in Python. This allows for e.g. cloud hosting providers to write their own tool to perform installations in their preferred programming language.
This introduces the concept of lockers which write the lock file, and installers which install from a lock file (they can be the same tool). The file format should promote good security defaults. As the format is not meant to be human-writable, this means having tools provide security-related details is reasonable and not a costly burden. The contents of a lock file should be able to replace the vast majority of uses of requirements files when used as a lock file (e.g. what pip-tools and pip freeze emit).
This means the file format specified by this PEP can, at minimum, act as an export target for tools which have their own internal lock file format. Lock files can be single-use and multi-use . Single-use lock files are things like requirements.txt files, which serve a single use-case/purpose (hence why it isn’t uncommon for a project to have multiple requirements files, each for a different use-case). Multi-use lock files represent multiple use-cases within a single file, often expressed through extras and Dependency Groups .
As such, this PEP supports additions to Environment Markers that allows for specifying extras and dependency groups as appropriate. This allows for a single lock file to support those cases. This not only cuts down on the number of potential lock files, but it also makes it easier when a single package is meant to be consistent across all use-cases (with multiple single-use lock files it requires coordinating across multiple lock files).
This support also means that this PEP supports all package installation-related data that can be recorded in a pyproject.toml file. The hope is that this support will allow some tools to drop their internal lock file entirely and solely rely on what this PEP specifies. Specification File Name A lock file MUST be named pylock.toml or match the regular expression
The rest of this page is on the live site.