PEP 633 – Dependency specification in pyproject.toml using an exploded TOML table | peps.python.org
PEP 633 – Dependency specification in pyproject.toml using an exploded TOML table PEP 633 – Dependency specification in pyproject.toml using an exploded TOML table Author : Laurie Opperman <laurie_opperman at hotmail.com>, Arun Babu Neelicattu <arun.neelicattu at gmail.com> Sponsor : Brett Cannon <brett at python.org> Discussions-To : Discourse thread Status : Rejected Type : Standards Track Topic : Packaging Created : 02-Sep-2020 Post-History : 02-Sep-2020 Resolution : Discourse message Table of Contents Rejection Notice Abstract Motivation Rationale Specification dependencies Requirement table optional-dependencies Reference implementation JSON schema Examples Compatibility Examples Version Constrained Dependencies Direct Reference Dependencies Environment Markers Package Extras Complex Examples Rejected Ideas Switch to an array for dependencies Replace optional-dependencies with extras direct table in requirement Include hash Dependency tables for each extra Environment marker keys Multiple extras which one requirement can satisfy Copyright Rejection Notice This PEP has been rejected in favour of PEP 631 due to its popularity, consistency with the existing usage of PEP 508 strings, and compatibility with existing packaging tool suites.
Abstract This PEP specifies how to write a project’s dependencies in a pyproject.toml file for packaging-related tools to consume using the fields defined in PEP 621 , as an alternative to the PEP 508 -based approach defined in PEP 631 . Motivation There are multiple benefits to using TOML tables and other data-types to represent requirements rather than PEP 508 strings: Easy initial validation via the TOML syntax. Easy secondary validation using a schema, for example a JSON Schema .
Potential for users to guess the keys of given features, rather than memorising a syntax. Users of multiple other popular languages may already be familiar with the TOML syntax. TOML directly represents the same data structures as in JSON, and therefore a sub-set of Python literals, so users can understand the hierarchy and type of value Rationale Most of this is taken from discussions in the PEP 621 dependencies topic .
This has elements from Pipfile , Poetry , Dart’s dependencies and Rust’s Cargo . A comparison document shows advantages and disadvantages between this format and PEP 508 -style specifiers. In the specification of multiple requirements with the same distribution name (where environment markers choose the appropriate dependency), the chosen solution is similar to Poetry ’s, where an array of requirements is allowed.
The direct-reference keys closely align with and utilise PEP 610 and PEP 440 as to reduce differences in the packaging ecosystem and rely on previous work in specification. Specification As in PEP 621 , if metadata is improperly specified then tools MUST raise an error. The metadata MUST conform to the TOML specification. To reduce confusion with this document being a specification for specifying dependencies, the word “requirement” is used to mean a PEP 508 dependency specification.
The following tables are added to the project table specified in PEP 621 . dependencies Format: table The keys inside this table are the names of the required distribution. The values can have one of the following types: string: the requirement is defined only by a version requirement, with same specification as version in the requirement table, except allowing the empty string "" to place no restriction on the version. table: a requirement table. array: an array of requirement tables.
It is an error to specify an empty array [] as a value. Requirement table The keys of the requirement table are as follows (all are optional): version (string): a PEP 440 version specifier, which is a comma-delimited list of version specifier clauses. The string MUST be non-empty. extras (array of strings): a list of PEP 508 extras declarations for the distribution. The list MUST be non-empty. markers (string): a PEP 508 environment marker expression.
The string MUST be non-empty. url (string): the URL of the artifact to install and satisfy the requirement. Note that file:// is the prefix used for packages to be retrieved from the local filesystem. git , hg , bzr or svn (string): the URL of a VCS repository (as specified in PEP 440 ) to clone, whose tree will be installed to satisfy the requirement. Further VCS keys will be added via amendments to PEP 610 , however tools MAY opt to support other VCS’s using their command-line command prior to the acceptance of the amendment. revision (string): the identifier for a specific revision of the specified VCS repository to check-out before installation.
Users MUST only provide this when one of git , hg , bzr , svn , or another VCS key is used to identify the distribution to install. Revision identifiers are suggested in PEP 610 . At most one of the following keys can be specified simultaneously, as they logically conflict with each other in the requirement: version , url , git , hg , bzr , svn , and any other VCS key. An empty requirement table {} places no restriction on the requirement, in addition to the empty string "" .
Any keys provided which are not specified in this document MUST cause an error in parsing. optional-dependencies Format: table The keys inside this table are the names of an extra’s required distribution. The values can have one of the following types: table: a requirement table. array: an array of requirement tables. These requirement tables have the same specification as above , with the addition of the following required key: for-extra (string): the name of the PEP 508 extra that this requirement is required for.
Reference implementation Tools will need to convert this format to PEP 508 requirement strings. Below is an example implementation of that conversion (assuming validation is already performed): def convert_requirement_to_pep508 ( name , requirement ): if isinstance ( requirement , str ): requirement = { "version" : requirement } pep508 = name if "extras" in requirement : pep508 += " [" + ", " . join ( requirement [ "extras" ]) + "]" if "version" in requirement : pep508 += " " + requirement [ "version" ] if "url" in requirement : pep508 += " @ " + requirement [ "url" ] for vcs in ( "git" , "hg" , "bzr" , "svn" ): if vcs in requirement : pep508 += " @ " + vcs + "+" + requirement [ vcs ] if "revision" in requirement : pep508 += "@" + requirement [ "revision" ] extra = None if "for-extra" in requirement : extra = requirement [ "for-extra" ] if "markers" in requirement : markers = requirement [ "markers" ] if extra : markers = "extra = '" + extra + "' and (" + markers + ")" pep508 += "; " + markers return pep508 , extra def convert_requirements_to_pep508 ( dependencies ): pep508s = [] extras = set () for name , req in dependencies . items (): if isinstance ( req , list ): for sub_req in req : pep508 , extra = convert_requirement_to_pep508 ( name , sub_req ) pep508s . append ( pep508 ) if extra : extras . add ( extra ) else : pep508 , extra = convert_requirement_to_pep508 ( name , req ) pep508s . append ( pep508 ) if extra : extras . add ( extra ) return pep508s , extras def convert_project_requirements_to_pep508 ( project ): reqs , _ = convert_requirements_to_pep508 ( project . get ( "dependencies" , {})) optional_reqs , extras = convert_requirements_to_pep508 ( project . get ( "optional-dependencies" , {}) ) reqs += optional_reqs return reqs , extras JSON schema For initial validation, a JSON-schema can be used.
Not only does this help tools have a consistent validation, but it allows code editors to highlight validation errors as users are building the dependencies list. { "$id" : "spam" , "$schema" : "http://json-schema.org/draft-07/schema#" , "title" : "Project metadata" , "type" : "object" , "definitions" : { "requirementTable" : { "title" : "Full project dependency specification" , "type" : "object" , "properties" : { "extras" : { "title" : "Dependenc
The rest of this page is on the live site.