Text of github.com › golang › go › discussions › 58409 as Styx crawled it on Nov 19, 2026. The page may have changed since. Only its words are kept, without pictures or layout, and only the first part of a long page.

Open the live page

telemetry in the Go toolchain · golang go · Discussion #58409 · GitHub

Skip to content You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert telemetry in the Go toolchain #58409 Closed Locked rsc announced in Discussions telemetry in the Go toolchain #58409 rsc Feb 8, 2023 · 95 comments · 394 replies Return to top Discussion options Uh oh!

There was an error while loading. Please reload this page . Uh oh! There was an error while loading. Please reload this page . rsc Feb 8, 2023 Feb 10 5pm US Eastern : Thanks for the lively discussion everyone. It has given me plenty to think about. We are starting to go in circles, and actually new comments are increasingly rare, so I've locked this to give everyone their weekend back. Thanks again. If anyone feels like there's something important that wasn't said in these 506 comments, please feel free to email me at rsc@golang.org .

Feb 26 The current plan is to use some form of opt-in to collect telemetry. How do software developers understand which parts of their software are being used and whether they are performing as expected? The modern answer is telemetry , which means software sending data to answer those questions back to a collection server. I believe that open-source software projects need to explore new telemetry designs that help developers get the information they need to work efficiently and effectively, without collecting invasive traces of detailed user activity.

I have written a short series of blog posts about one such design, which I call transparent telemetry , because it collects as little as possible (kilobytes per year from each installation) and then publishes every bit that it collects, for public inspection and analysis. I’d like to explore using transparent telemetry, or a system like it, in the Go toolchain, which I hope will help Go project developers and users alike.

To be clear, I am only suggesting that the instrumentation be added to the Go command-line tools written and distributed by the Go team, such as the go command, the Go compiler, gopls , and govulncheck . I am not suggesting that instrumentation be added by the Go compiler to all Go programs in the world: that’s clearly inappropriate. Transparent telemetry has the following key properties: The decisions about what metrics to collect are made in an open, public process.

The collection configuration is automatically generated from the actively tracked metrics: no data is collected that isn’t needed for the metrics. The collection configuration is served using a tamper-evident transparent log, making it very difficult to serve different collection configurations to different systems. The collection configuration is a cacheable, proxied Go module, so any privacy-enhancing local Go proxy already in use for ordinary modules will automatically be used for collection configuration.

To further ameliorate concerns about tracking systems by the downloading of the collection configuration, each installation only bothers downloading the configuration each week with probability 10%, so that each installation only asks for the configuration about five times per year. Uploaded reports only include total event counts over a full week, not any kind of time-ordered event trace. Uploaded reports do not include user IDs, machine IDs, or any other kind of ID.

Uploaded reports only contain strings that are already known to the collection server: counter names, program names, and version strings repeated from the collection configuration, along with the names of functions in specific, unmodified Go toolchain programs for stack traces. The only types of non-string data in the reports are event counts, dates, and line numbers. IP addresses exposed by the HTTP session that uploads the report are not recorded with the reports.

Thanks to sampling , only a constant number of uploaded reports are needed to achieve a specific accuracy target, no matter how many installations exist. Specifically, only about 16,000 reports are needed for 1% accuracy at a 99% confidence level. This means that as new systems are added to the system, each system reports less often . With a conservative estimate of two million Go installations, about 16,000 reporting each week corresponds to an overall reporting rate of well under 2% per week, meaning each installation would upload a report on average less than once per year.

The aggregate computed metrics are made public in graphical and tabular form. The full raw data as collected is made public, so that project maintainers have no proprietary advantage or insights in their role as the direct data collector. The system is on by default, but opting out is easy, effective, and persistent. Please read the introductory blog post for more background before commenting here. You may also want to check out the detailed design and list of use cases .

You must be logged in to vote All reactions 👍 159 👎 507 😄 4 😕 50 ❤️ 40 🚀 4 👀 6 Replies: 95 comments · 394 replies Comment options Uh oh! There was an error while loading. Please reload this page . Uh oh! There was an error while loading. Please reload this page . Stolas Feb 8, 2023 Should be off by default. There is no reason for a development chain to have any kind of telemetry on by default. Or, as I do understand the need for it to be on by default, it should check what the user wants on first start.

One reason it should be off by default would be cases such as Jenkins farms, Bazel use, and other locations that arnt your regular development environment. You must be logged in to vote All reactions 👍 282 👎 23 😕 5 23 replies Comment options Uh oh! There was an error while loading. Please reload this page . w3bb Feb 9, 2023 I think that lowering the barrier to opt-in and share data is a better solution than increasing the barrier to opt out.

People who would decline telemetry at a prompt are people who don't want telemetry, making the life of those people harder is really bad whether it's intentional or not. All reactions 👍 5 This comment was marked as spam. Sign in to view Comment options Uh oh! There was an error while loading. Please reload this page . padawan Feb 9, 2023 As soon as there is just one identifiable data collected, opt-out becomes completely illegal WRT GDPR, which applies to any entity processing a European user data regardless where it is collected.

Add that the privacy shield is illegal now in Europe, you better watch out for the huge fine if that ships with opt-out to any European user. This is a legal issue, your personal opinion on the goods and bads of opt-in vs opt-out is irrelevant. All reactions 👍 18 😕 2 Comment options Uh oh! There was an error while loading. Please reload this page . Uh oh! There was an error while loading. Please reload this page . doggedOwl Feb 10, 2023 Transparent and "On by default" do not go together.

Naming It transparent is like the 1984 moto "war is peace". It is often said here that just because this is discussed and can be checked publicly is enough and noone should be taken offguard. “There’s no point in acting surprised about it. All the planning charts and demolition orders have been on display at your local planning department in Alpha Centauri for 50 of your Earth years, so you’ve had plenty of time to lodge any formal complaint and it’s far too late to start making a fuss about it now....

I’m sorry, but if you can’t be bothered to take an interest in local affairs, that’s your own lookout. " Vogon Transparency. All reactions 👍 5 Comment options Uh oh! There was an error while loading. Please reload this page . Uh oh! There was an error while loading. Please reload this page . gophun Feb 10, 2023 I wrote a little about this at https://research.swtch.com/telemetry-design#opt-out . [...] In systems that have collection off by default, opt-in rates

The rest of this page is on the live site.