PEP 474 – Creating forge.python.org | peps.python.org
PEP 474 – Creating forge.python.org PEP 474 – Creating forge.python.org Author : Alyssa Coghlan <ncoghlan at gmail.com> Status : Withdrawn Type : Process Created : 19-Jul-2014 Post-History : 19-Jul-2014, 08-Jan-2015, 01-Feb-2015 Table of Contents Abstract PEP Withdrawal Proposal Rationale Intended Benefits Sustaining Engineering Considerations Personal Motivation Technical Concerns and Challenges Service hosting Ongoing infrastructure maintenance User account management Breaking existing SSH access and links for Mercurial repositories Integration with Roundup Accepting pull requests on GitHub and BitBucket Transparent Git and Mercurial interoperability Pilot Objectives and Timeline Future Implications for CPython Core Development Copyright Abstract This PEP proposes setting up a new PSF provided resource, forge.python.org, as a location for maintaining various supporting repositories (such as the repository for Python Enhancement Proposals) in a way that is more accessible to new contributors, and easier to manage for core developers.
This PEP does not propose any changes to the core development workflow for CPython itself (see PEP 462 in relation to that). PEP Withdrawal This PEP has been withdrawn by the author in favour of the GitLab based proposal in PEP 507 . If anyone else would like to take over championing this PEP, contact the core-workflow mailing list Proposal This PEP proposes that an instance of the self-hosted Kallithea code repository management system be deployed as “forge.python.org”.
Individual repositories (such as the developer guide or the PEPs repository) may then be migrated from the existing hg.python.org infrastructure to the new forge.python.org infrastructure on a case-by-case basis. Each migration will need to decide whether to retain a read-only mirror on hg.python.org, or whether to just migrate wholesale to the new location. In addition to supporting read-only mirrors on hg.python.org, forge.python.org will also aim to support hosting mirrors on popular proprietary hosting sites like GitHub and BitBucket.
The aim will be to allow users familiar with these sites to submit and discuss pull requests using their preferred workflow, with forge.python.org automatically bringing those contributions over to the master repository. Given the availability and popularity of commercially backed “free for open source projects” repository hosting services, this would not be a general purpose hosting site for arbitrary Python projects.
The initial focus will be specifically on CPython and other repositories currently hosted on hg.python.org. In the future, this could potentially be expanded to consolidating other PSF managed repositories that are currently externally hosted to gain access to a pull request based workflow, such as the repository for the python.org Django application. As with the initial migrations, any such future migrations would be considered on a case-by-case basis, taking into account the preferences of the primary users of each repository.
Rationale Currently, hg.python.org hosts more than just the core CPython repository, it also hosts other repositories such as those for the CPython developer guide and for Python Enhancement Proposals, along with various “sandbox” repositories for core developer experimentation. While the simple “pull request” style workflow made popular by code hosting sites like GitHub and BitBucket isn’t adequate for the complex branching model needed for parallel maintenance and development of the various CPython releases, it’s a good fit for several of the ancillary projects that surround CPython that we don’t wish to move to a proprietary hosting site.
The key requirements proposed for a PSF provided software forge are: MUST support simple “pull request” style workflows MUST support online editing for simple changes MUST be backed by an active development organisation (community or commercial) MUST support self-hosting of the master repository on PSF infrastructure without ongoing fees Additional recommended requirements that are satisfied by this proposal, but may be negotiable if a sufficiently compelling alternative is presented: SHOULD be a fully open source application written in Python SHOULD support Mercurial (for consistency with existing tooling) SHOULD support Git (to provide that option to users that prefer it) SHOULD allow users of git and Mercurial clients to transparently collaborate on the same repository SHOULD allow users of GitHub and BitBucket to submit proposed changes using the standard pull request workflows offered by those tools SHOULD be open to customisation to meet the needs of CPython core development, including providing a potential path forward for the proposed migration to a core reviewer model in PEP 462 The preference for self-hosting without ongoing fees rules out the free-as-in-beer providers like GitHub and BitBucket, in addition to the various proprietary source code management offerings.
The preference for Mercurial support not only rules out GitHub, but also other Git-only solutions like GitLab and Gitorious. The hard requirement for online editing support rules out the Apache Allura/HgForge combination. The preference for a fully open source solution rules out RhodeCode. Of the various options considered by the author of this proposal, that leaves Kallithea SCM as the proposed foundation for a forge.python.org service.
Kallithea is a full GPLv3 application (derived from the clearly and unambiguously GPLv3 licensed components of RhodeCode) that is being developed under the auspices of the Software Freedom Conservancy . The Conservancy has affirmed that the Kallithea codebase is completely and validly licensed under GPLv3. In addition to their role in building the initial Kallithea community, the Conservancy is also the legal home of both the Mercurial and Git projects.
Other SFC member projects that may be familiar to Python users include Twisted, Gevent, BuildBot and PyPy. Intended Benefits The primary benefit of deploying Kallithea as forge.python.org is that supporting repositories such as the developer guide and the PEP repo could potentially be managed using pull requests and online editing. This would be much simpler than the current workflow which requires PEP editors and other core developers to act as intermediaries to apply updates suggested by other users.
The richer administrative functionality would also make it substantially easier to grant users access to particular repositories for collaboration purposes, without having to grant them general access to the entire installation. This helps lower barriers to entry, as trust can more readily be granted and earned incrementally, rather than being an all-or-nothing decision around granting core developer access. Sustaining Engineering Considerations Even with its current workflow, CPython itself remains one of the largest open source projects in the world (in the top 2% of projects tracked on OpenHub).
Unfortunately, we have been significantly less effective at encouraging contributions to the projects that make up CPython’s workflow infrastructure, including ensuring that our installations track upstream, and that wherever feasible, our own customisations are contributed back to the original project. As such, a core component of this proposal is to actively engage with the upstream Kallithea community to lower the barriers to working with and on the Kallithea SCM, as well as with the PSF Infrastructure team to ensure the forge.python.org service integrates cleanly with the PSF’s infrastructure automation.
This approach aims to provide a number of key benefits: allowing those of us contributing to maintenance of this service to be as productive as possible in the time we have available offering a compelling professional development opportunity to those volunteers that choose to participate in maintenance of this service making the Kallithea project itself mo
The rest of this page is on the live site.