PEP 481 – Migrate CPython to Git, Github, and Phabricator | peps.python.org
PEP 481 – Migrate CPython to Git, Github, and Phabricator PEP 481 – Migrate CPython to Git, Github, and Phabricator Author : Donald Stufft <donald at stufft.io> Status : Withdrawn Type : Process Created : 29-Nov-2014 Post-History : 29-Nov-2014 Table of Contents Abstract Rationale Version Control System Repository Hosting Code Review GitHub Pull Requests Phabricator Criticism X is not written in Python GitHub is not Free/Open Source Mercurial is better than Git CPython Workflow is too Complicated Example: Scientific Python References Copyright Abstract Note This PEP has been withdrawn, if you’re looking for the PEP documenting the move to Github, please refer to PEP 512 .
This PEP proposes migrating the repository hosting of CPython and the supporting repositories to Git and Github. It also proposes adding Phabricator as an alternative to Github Pull Requests to handle reviewing changes. This particular PEP is offered as an alternative to PEP 474 and PEP 462 which aims to achieve the same overall benefits but restricts itself to tools that support Mercurial and are completely Open Source.
Rationale CPython is an open source project which relies on a number of volunteers donating their time. As an open source project it relies on attracting new volunteers as well as retaining existing ones in order to continue to have a healthy amount of manpower available. In addition to increasing the amount of manpower that is available to the project, it also needs to allow for effective use of what manpower is available.
The current toolchain of the CPython project is a custom and unique combination of tools which mandates a workflow that is similar to one found in a lot of older projects, but which is becoming less and less popular as time goes on. The one-off nature of the CPython toolchain and workflow means that any new contributor is going to need spend time learning the tools and workflow before they can start contributing to CPython.
Once a new contributor goes through the process of learning the CPython workflow they also are unlikely to be able to take that knowledge and apply it to future projects they wish to contribute to. This acts as a barrier to contribution which will scare off potential new contributors. In addition the tooling that CPython uses is under-maintained, antiquated, and it lacks important features that enable committers to more effectively use their time when reviewing and approving changes.
The fact that it is under-maintained means that bugs are likely to last for longer, if they ever get fixed, as well as it’s more likely to go down for extended periods of time. The fact that it is antiquated means that it doesn’t effectively harness the capabilities of the modern web platform. Finally the fact that it lacks several important features such as a lack of pre-testing commits and the lack of an automatic merge tool means that committers have to do needless busy work to commit even the simplest of changes.
Version Control System The first decision that needs to be made is the VCS of the primary server side repository. Currently the CPython repository, as well as a number of supporting repositories, uses Mercurial. When evaluating the VCS we must consider the capabilities of the VCS itself as well as the network effect and mindshare of the community around that VCS. There are really only two real options for this, Mercurial and Git.
Between the two of them the technical capabilities are largely equivalent. For this reason this PEP will largely ignore the technical arguments about the VCS system and will instead focus on the social aspects. It is not possible to get exact numbers for the number of projects or people which are using a particular VCS, however we can infer this by looking at several sources of information for what VCS projects are using.
The Open Hub (previously Ohloh) statistics [1] show that 37% of the repositories indexed by The Open Hub are using Git (second only to SVN which has 48%) while Mercurial has just 2% (beating only bazaar which has 1%). This has Git being just over 18 times as popular as Mercurial on The Open Hub. Another source of information on the popular of the difference VCSs is PyPI itself. This source is more targeted at the Python community itself since it represents projects developed for Python.
Unfortunately PyPI does not have a standard location for representing this information, so this requires manual processing. If we limit our search to the top 100 projects on PyPI (ordered by download counts) we can see that 62% of them use Git while 22% of them use Mercurial while 13% use something else. This has Git being just under 3 times as popular as Mercurial for the top 100 projects on PyPI. Obviously from these numbers Git is by far the more popular DVCS for open source projects and choosing the more popular VCS has a number of positive benefits.
For new contributors it increases the likelihood that they will have already learned the basics of Git as part of working with another project or if they are just now learning Git, that they’ll be able to take that knowledge and apply it to other projects. Additionally a larger community means more people writing how to guides, answering questions, and writing articles about Git which makes it easier for a new user to find answers and information about the tool they are trying to learn.
Another benefit is that by nature of having a larger community, there will be more tooling written around it. This increases options for everything from GUI clients, helper scripts, repository hosting, etc. Repository Hosting This PEP proposes allowing GitHub Pull Requests to be submitted, however GitHub does not have a way to submit Pull Requests against a repository that is not hosted on GitHub. This PEP also proposes that in addition to GitHub Pull Requests Phabricator’s Differential app can also be used to submit proposed changes and Phabricator does allow submitting changes against a repository that is not hosted on Phabricator.
For this reason this PEP proposes using GitHub as the canonical location of the repository with a read-only mirror located in Phabricator. If at some point in the future GitHub is no longer desired, then repository hosting can easily be moved to solely in Phabricator and the ability to accept GitHub Pull Requests dropped. In addition to hosting the repositories on Github, a read only copy of all repositories will also be mirrored onto the PSF Infrastructure.
Code Review Currently CPython uses a custom fork of Rietveld which has been modified to not run on Google App Engine which is really only able to be maintained currently by one person. In addition it is missing out on features that are present in many modern code review tools. This PEP proposes allowing both Github Pull Requests and Phabricator changes to propose changes and review code. It suggests both so that contributors can select which tool best enables them to submit changes, and reviewers can focus on reviewing changes in the tooling they like best.
GitHub Pull Requests GitHub is a very popular code hosting site and is increasingly becoming the primary place people look to contribute to a project. Enabling users to contribute through GitHub is enabling contributors to contribute using tooling that they are likely already familiar with and if they are not they are likely to be able to apply to another project. GitHub Pull Requests have a fairly major advantage over the older “submit a patch to a bug tracker” model.
It allows developers to work completely within their VCS using standard VCS tooling so it does not require creating a patch file and figuring out what the right location is to upload it to. This lowers the barrier for sending a change to be reviewed. On the reviewing side, GitHub Pull Requests are far easier to review, they have nice syntax highlighted diffs which can operate in either unified or side by side views. They allow expanding the context on a dif
The rest of this page is on the live site.