Moving LLVM Projects to GitHub - LLVM
Hide navigation sidebar Hide table of contents sidebar Back to top Edit this page Toggle table of contents sidebar Moving LLVM Projects to GitHub # Current Status # We are planning to complete the transition to GitHub by Oct 21, 2019. See the GitHub migration status page for the latest updates and instructions for how to migrate your workflows. Introduction # This is a proposal to move our current revision control system from our own hosted Subversion to GitHub.
Below are the financial and technical arguments as to why we are proposing such a move and how people (and validation infrastructure) will continue to work with a Git-based LLVM. What This Proposal is Not About # Changing the development policy. This proposal relates only to moving the hosting of our source-code repository from SVN hosted on our own servers to Git hosted on GitHub. We are not proposing using GitHub’s issue tracker, pull-requests, or code-review.
Contributors will continue to earn commit access on demand under the Developer Policy, except that that a GitHub account will be required instead of SVN username/password-hash. Why Git, and Why GitHub? # Why Move At All? # This discussion began because we currently host our own Subversion server and Git mirror on a voluntary basis. The LLVM Foundation sponsors the server and provides limited support, but there is only so much it can do.
Volunteers are not sysadmins themselves, but compiler engineers that happen to know a thing or two about hosting servers. We also don’t have 24/7 support, and we sometimes wake up to see that continuous integration is broken because the SVN server is either down or unresponsive. We should take advantage of one of the services out there (GitHub, GitLab, and BitBucket, among others) that offer better service (24/7 stability, disk space, Git server, code browsing, forking facilities, etc) for free.
Why Git? # Many new coders nowadays start with Git, and a lot of people have never used SVN, CVS, or anything else. Websites like GitHub have changed the landscape of open source contributions, reducing the cost of first contribution and fostering collaboration. Git is also the version control many LLVM developers use. Despite the sources being stored in a SVN server, these developers are already using Git through the Git-SVN integration.
Git allows you to: Commit, squash, merge, and fork locally without touching the remote server. Maintain local branches, enabling multiple threads of development. Collaborate on these branches (e.g. through your own fork of llvm on GitHub). Inspect the repository history (blame, log, bisect) without Internet access. Maintain remote forks and branches on Git hosting services and integrate back to the main repository. In addition, because Git seems to be replacing many OSS projects’ version control systems, there are many tools that are built over Git.
Future tooling may support Git first (if not only). Why GitHub? # GitHub, like GitLab and BitBucket, provides free code hosting for open source projects. Any of these could replace the code-hosting infrastructure that we have today. These services also have a dedicated team to monitor, migrate, improve and distribute the contents of the repositories depending on region and load. GitHub has one important advantage over GitLab and BitBucket: it offers read-write SVN access to the repository ( https://github.com/blog/626-announcing-svn-support ).
This would enable people to continue working post-migration as though our code were still canonically in an SVN repository. In addition, there are already multiple LLVM mirrors on GitHub, indicating that part of our community has already settled there. On Managing Revision Numbers with Git # The current SVN repository hosts all the LLVM sub-projects alongside each other. A single revision number (e.g. r123456) thus identifies a consistent version of all LLVM sub-projects.
Git does not use sequential integer revision number but instead uses a hash to identify each commit. The loss of a sequential integer revision number has been a sticking point in past discussions about Git: “The ‘branch’ I most care about is mainline, and losing the ability to say ‘fixed in r1234’ (with some sort of monotonically increasing number) would be a tragic loss.” [ 1 ] “I like those results sorted by time and the chronology should be obvious, but timestamps are incredibly cumbersome and make it difficult to verify that a given checkout matches a given set of results.” [ 2 ] “There is still the major regression with unreadable version numbers.
Given the amount of Bugzilla traffic with ‘Fixed in…’, that’s a non-trivial issue.” [ 3 ] “Sequential IDs are important for LNT and llvmlab bisection tool.” [ 4 ] . However, Git can emulate this increasing revision number: git rev-list --count <commit-hash> . This identifier is unique only within a single branch, but this means the tuple (num, branch-name) uniquely identifies a commit. We can thus use this revision number to ensure that e.g. clang -v reports a user-friendly revision number (e.g. main-12345 or 4.0-5321 ), addressing the objections raised above with respect to this aspect of Git.
What About Branches and Merges? # In contrast to SVN, Git makes branching easy. Git’s commit history is represented as a DAG, a departure from SVN’s linear history. However, we propose to mandate making merge commits illegal in our canonical Git repository. Unfortunately, GitHub does not support server side hooks to enforce such a policy. We must rely on the community to avoid pushing merge commits. GitHub offers a feature called Status Checks : a branch protected by status checks requires commits to be explicitly allowed before the push can happen.
We could supply a pre-push hook on the client side that would run and check the history, before allowing the commit being pushed [ 5 ] . However this solution would be somewhat fragile (how do you update a script installed on every developer machine?) and prevents SVN access to the repository. What About Commit Emails? # We will need a new bot to send emails for each commit. This proposal leaves the email format unchanged besides the commit URL.
Straw Man Migration Plan # Step #1 : Before The Move # Update docs to mention the move, so people are aware of what is going on. Set up a read-only version of the GitHub project, mirroring our current SVN repository. Add the required bots to implement the commit emails, as well as the umbrella repository update (if the multirepo is selected) or the read-only Git views for the sub-projects (if the monorepo is selected).
Step #2 : Git Move # Update the buildbots to pick up updates and commits from the GitHub repository. Not all bots have to migrate at this point, but it’ll help provide infrastructure testing. Update Phabricator to pick up commits from the GitHub repository. LNT and llvmlab have to be updated: they rely on unique monotonically increasing integer across branch [ 4 ] . Instruct downstream integrators to pick up commits from the GitHub repository.
Review and prepare an update for the LLVM documentation. Until this point nothing has changed for developers, it will just boil down to a lot of work for buildbot and other infrastructure owners. The migration will pause here until all dependencies have cleared, and all problems have been solved. Step #3: Write Access Move # Collect developers’ GitHub account information, and add them to the project. Switch the SVN repository to read-only and allow pushes to the GitHub repository.
Update the documentation. Mirror Git to SVN. Step #4 : Post Move # Archive the SVN repository. Update links on the LLVM website pointing to viewvc/klaus/phab etc. to point to GitHub instead. GitHub Repository Description # Monorepo # The LLVM git repository hosted at https://github.com/llvm/llvm-project contains all sub-projects in a single source tree. It is often referred to as a monorepo and mimics an export of the curr
The rest of this page is on the live site.