PEP 804 – An external dependency registry and name mapping mechanism | peps.python.org
PEP 804 – An external dependency registry and name mapping mechanism PEP 804 – An external dependency registry and name mapping mechanism Author : Pradyun Gedam <pradyunsg at gmail.com>, Ralf Gommers <ralf.gommers at gmail.com>, Michał Górny <mgorny at quansight.com>, Jaime Rodríguez-Guerra <jaime.rogue at gmail.com>, Michael Sarahan <msarahan at gmail.com> Discussions-To : Discourse thread Status : Draft Type : Standards Track Topic : Packaging Requires : 725 Created : 03-Sep-2025 Post-History : 22-Sep-2025 Table of Contents Abstract Motivation Rationale Prior art Governance and maintenance costs of name mappings Generating package manager-specific install commands Registry design Specification Central registry Schema $schema schema_version definitions Known ecosystems Schema $schema schema_version ecosystems Mappings Schema $schema schema_version name description mappings package_managers commands specifier_syntax Redistribution Examples Registry Known ecosystems Mappings Practical examples pyproject-external CLI pyproject-external API Grayskull Backwards Compatibility Security Implications How to Teach This Central DepURL registry maintainers Package ecosystem maintainers usage Maintainers of Python projects End user package consumers Reference Implementation Rejected Ideas Centralized mappings governed by the same body Allowing ecosystem-specific variants of packages Adding more package metadata to the central registry Mapping PyPI projects to repackaged counterparts in target ecosystems Strict validation of identifiers Inheritance and cross-referenced mappings Tracking package name changes Reusing existing databases as a central registry Open Issues References Appendix A: Operational suggestions Appendix B: Virtual versioning proposal Copyright Abstract This PEP specifies a name mapping mechanism that allows packaging tools to map external dependency identifiers (as introduced in PEP 725 ) to their counterparts in other package repositories.
Motivation Packages on PyPI often require build-time and runtime dependencies that are not present on PyPI. PEP 725 introduced metadata to express such dependencies. Using concrete external dependency metadata for a Python package requires mapping the given dependency identifiers to the specifiers used in other ecosystems, which would allow: Enabling tools to automatically map external dependencies to packages in other packaging repositories/ecosystems, Including the needed external dependencies with the package names used by the relevant system package manager on the user’s system in error messages emitted by Python package installers and build frontends, as well as allowing the user to obtain installation instructions for those packages.
Packaging ecosystems like Linux distros, conda, Homebrew, Spack, and Nix need full sets of dependencies for Python packages, and have tools like pyp2rpm (Fedora), Grayskull (conda), and dh_python (Debian) which attempt to automatically generate dependency information from the metadata available in upstream Python packages. Before PEP 725, external dependencies were handled manually, because there was no metadata for this in pyproject.toml or any other standard metadata file.
Enabling its automatic conversion is a key benefit of this PEP, making Python packaging easier and more reliable. In addition, the authors envision other types of tools making use of this information; e.g. dependency analysis tools like Repology , Dependabot and libraries.io . Rationale Prior art The R language has a System Requirements for R packages with a central registry that knows how to translate external dependency metadata to install commands for package managers like apt-get .
This registry centralises the mappings for a series of Linux distributions, and also Windows. macOS is not present. The “Rule Coverage” of its README used to show that this system improves the chance of success of building packages from CRAN from source. Across all CRAN packages, Ubuntu 18 improved from 78.1% to 95.8%, CentOS 7 from 77.8% to 93.7% and openSUSE 15.0 from 78.2% to 89.7%. The chance of success depends on how well the registry is maintained, but the gain is significant: ~4x fewer packages fail to build on Ubuntu and CentOS in a Docker container.
RPM-based distributions, like Fedora, can use a rule-based implementation ( NameConvertor ) in pyp2rpm . The main rule is that the RPM name for a PyPI package is typically f"python3-{pypi_package_name}" . The rare exceptions include packages that primarily distribute an application, which drop the prefix, (e.g. the Black formatter is simply black , not python3-black ), and variants for different Python versions (e.g. in RHEL 9 setuptools can be found as python3-setuptools for Python 3.9, but python3.11-setuptools and python3.12-setuptools are also available).
More details are available in Fedora’s packaging guidelines for Python . Debian packages typically follow a f"python3-{import_name}" naming scheme, with some exceptions: some sub-communities have an infix (e.g. Django packages go under f"python3-django-*" ), and applications are often distributed by their name, with no python3- prefix. Additional details are available in Debian’s Python Policy . Gentoo follows a similar approach to naming Python packages, using the dev-python/ category and some well-specified rules .
Conda-forge has a more explicit name mapping, because the base names are the same in conda-forge as on PyPI (e.g. numpy maps to numpy ), but there are many exceptions because of both name collisions and renames (e.g. the PyPI name for PyTorch is torch while in conda-forge it’s pytorch ). There are several name mappings efforts maintained by different teams. Conda-forge’s infrastructure generates one in regro/cf-graph-countyfair .
Grayskull maintains its own curated mapping . Prefix.dev created the parselmouth mappings to support conda and PyPI integrations in their tooling. A more complete overview of their approaches, strengths and weaknesses can be found in conda/grayskull#564 . The OpenStack ecosystem also needs to deal with some mapping efforts. All of them focus on Linux distributions, exclusively. pkg-map accompanies diskimage-builder and provides a file format where the user defines arbitrary variable names and their corresponding names in the target distro (Red Hat, Debian, OpenSUSE, etc).
See example for PyYAML . bindep defines a file bindep.txt (see example ) where users can write down dependencies that are not installable from PyPI. The format is line-based, with each line containing a dependency as found in the Debian ecosystem. For other distributions, it offers a “filters” syntax between square brackets where users can indicate other target platforms, optional dependencies and extras. The need for mappings is also found in other ecosystems like SageMath , but also by end-users themselves who want to install PyPI packages with their system package manager of choice ( example StackOverflow question ).
Governance and maintenance costs of name mappings The maintenance cost of external dependency mappings to a large number of packaging ecosystems is potentially high. We choose to define the registry in such a way that: A central authority maintains the list of recognized DepURLs and the known ecosystem mappings. The mappings themselves are maintained by the target packaging ecosystems. Hence this system is opt-in for a given ecosystem, and the associated maintenance costs are distributed.
Generating package manager-specific install commands Python package authors with external dependencies usually have installation instructions for those external dependencies in their documentation. These instructions are difficult to write and keep up-to-date, and are usually only covering one or at most a handful of platforms. As an example, here are SciPy’s instructions for its external build dependencies (C/C++/Fortran compilers, OpenBLAS, pkg-config): Debian/Ubuntu: sudo ap
The rest of this page is on the live site.