Go Wiki: DashboardBuilders - The Go Programming Language
Go Wiki: DashboardBuilders LUCI Builders The Go team has migrated the testing pipeline from a custom solution, the coordinator, to LUCI . LUCI is an open source continuous integration system created by the Chrome open source team at Google. The Go team has adopted the use of LUCI in order to leverage a continuous integration solution which is used and supported by a larger group of developers. This should enable the team to provide a more featureful solution to the community.
The LUCI system requires builders to run two applications which authenticate to LUCI and receive and process builds. LUCI token daemon generates a token needed to authenticate. The swarming bot uses the token to connect to LUCI and process builds. Builder Requirements An internet connection with the ability to connect to: https://proxy.golang.org (or an alternative proxy via GOPROXY). https://luci-token-server.appspot.com https://chromium-swarm.appspot.com https://cr-buildbucket.appspot.com https://7419-34ac013-dot-chromium-swarm.appspot.com (possible changed in the future?) https://remotebuildexecution.googleapis.com https://storage.googleapis.com (for gomote creation ) Resources At least 512MB of memory.
1GB or more is highly recommended. 20GB disk space is ideal. Preferably with 2 or more (V)CPUs. Python 3 installed and available in PATH (or equivalent) under the name python3 (or python3.exe on Windows). Any reasonably modern version of Python 3 should work. Some versions of Python 3 that are too old or too new may not be as well-tested, and you may uncover problems. Consider looking up the exact Python 3 versions already being used in the luci.golang.ci-workers pool (you may be prompted to sign in to view that page), as those are certain to be well-tested.
Permissions The bot should be run as the swarming user (without root rights). The bot automatically updates itself. It should have permissions to do so. The bot periodically restarts the machine. It should have permissions to do so (via sudo). Under Docker, you can replace the shutdown command with a shell script that restarts the container ( example ). If the machine can’t be restarted for some reason, set the environment variable SWARMING_NEVER_REBOOT .
How to set up a builder Create an issue on the Go Issue tracker requesting the addition of a new builder and assign it yourself. The title of the issue should be in the format: x/build: add LUCI <os-arch> builder . Choose a hostname and state its value in the issue body. The hostname should follow the following format: <GOOS>-<GOOARCH>-<GitHub handle of maintainer> . The Go team may ask that it be changed if there is any conflict with the name.
Add the label “new-builder”. (You can post a comment on the issue stating @gopherbot, please add label new-builder. in the issue to have gopherbot add it for you.) Use golang.org/x/build/cmd/genbotcert to generate both a certificate signing request ( hostname .csr) and a TLS private key ( hostname .key) using the hostname (chosen beforehand) as input. Add a .txt file extension to the certificate signing request ( hostname .csr.txt) and attach it to the GitHub issue.
A team member will attach the resulting certificate ( hostname .cert) to the GitHub issue. genbotcert -bot-hostname <hostname> A Go team member will define your new builder in LUCI . A comment will be added to the issue when this is completed. The Machine Token Daemon communicates with the Token Server to generate and renew a LUCI machine token. Install go.chromium.org/luci/tokenserver/cmd/luci_machine_tokend and configure it to run every 10 minutes via cron.
The private key shouldn’t be readable by the swarming user, so the cron job should run as a separate user. luci_machine_tokend -backend luci-token-server.appspot.com -cert-pem <path-to-the-certificate> -pkey-pem <path-to-the-private-key> -token-file=/var/lib/luci_machine_tokend/token.json If /var/lib isn’t a suitable place for the token, change it as you see fit and set the environment variable LUCI_MACHINE_TOKEN to the file path when calling bootstrapswarm below.
Install golang.org/x/build/cmd/bootstrapswarm and configure it to run in a loop under your operating system’s process supervisor (systemd, etc) as the swarming user. Bootstrapswarm downloads the initial version of the swarming bot and ensures that it is always running. bootstrapswarm -hostname <hostname> Verify the bot starts up without any errors in the logs. Security notes Generally, low-capacity builders only run code that’s already been reviewed & submitted (post-submit testing).
We only enable pre-submit testing for builders run by the Go team that have a lot of hardware available. However, the Gomote tool is available for a number of people on the Go team and in the Go community that lets them have arbitrary access to the builders for development & debugging. For paranoia reasons, you might want to run your builder in an isolated network that can’t access any of your internal resources. This content is part of the Go Wiki . go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic.
Learn more.