redefining for loop variable semantics · golang go · Discussion #56010 · GitHub
Skip to content You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert redefining for loop variable semantics #56010 Closed Locked rsc announced in Discussions redefining for loop variable semantics #56010 rsc Oct 3, 2022 · 50 comments · 241 replies Return to top Discussion options Uh oh!
There was an error while loading. Please reload this page . Uh oh! There was an error while loading. Please reload this page . rsc Oct 3, 2022 Update 2023-06-06: Go 1.21 is expected to support GOEXPERIMENT=loopvar as a way of trying out these new semantics. See #57969 and https://go.dev/wiki/LoopvarExperiment . We have been looking at what to do about the for loop variable problem ( #20733 ), gathering data about what a change would mean and how we might deploy it.
This discussion aims to gather early feedback about this idea, to understand concerns and aspects we have not yet considered. Thanks for keeping this discussion respectful and productive! To recap #20733 briefly, the problem is that loops like this one don’t do what they look like they do: var all []*Item for _, item := range items { all = append(all, &item) } That is, this code has a bug. After this loop executes, all contains len(items) identical pointers, each pointing at the same Item , holding the last value iterated over.
This happens because the item variable is per-loop, not per-iteration: &item is the same on every iteration, and item is overwritten on each iteration. The usual fix is to write this instead: var all []*Item for _, item := range items { item := item all = append(all, &item) } This bug also often happens in code with closures that capture the address of item implicitly, like: var prints []func() for _, v := range []int{1, 2, 3} { prints = append(prints, func() { fmt.Println(v) }) } for _, print := range prints { print() } This code prints 3, 3, 3, because all the closures print the same v, and at the end of the loop, v is set to 3.
Note that there is no explicit &v to signal a potential problem. Again the fix is the same: add v := v. Goroutines are also often involved, although as these examples show, they need not be. See also the Go FAQ entry . We have talked for a long time about redefining these semantics, to make loop variables per-iteration instead of per-loop . That is, the change would effectively be to add an implicit “x := x” at the start of every loop body for each iteration variable x, just like people do manually today.
Making this change would remove the bugs from the programs above. In the Go 2 transitions document we gave the general rule that language redefinitions like what I just described are not permitted. I believe that is the right general rule, but I have come to also believe that the for loop variable case is strong enough to motivate a one-time exception to that rule. Loop variables being per-loop instead of per-iteration is the only design decision I know of in Go that makes programs incorrect more often than it makes them correct.
Since it is the only such design decision, I do not see any plausible candidates for additional exceptions. To make the breakage completely user controlled, the way the rollout would work is to change the semantics based on the go line in each package’s go.mod file, the same line we already use for enabling language features (you can only use generics in packages whose go.mod says “go 1.18” or later). Just this once, we would use the line for changing semantics instead of for adding a feature or removing a feature.
If we hypothetically made the change in go 1.30, then modules that say “go 1.30” or later get the per-iteration variables, while modules with earlier versions get the per-loop variables: In a given code base, the change would be “gradual” in the sense that each module can update to the new semantics independently, avoiding a bifurcation of the ecosystem. The specific semantics of the redefinition would be that both range loops and three-clause for loops get per-iteration variables.
So in addition to the program above being fixed, this one would be fixed too: var prints []func() for i := 1; i <= 3; i++ { prints = append(prints, func() { fmt.Println(i) }) } for _, print := range prints { print() } In the 3-clause form, the start of the iteration body copies the per-loop i into a per-iteration i , and then the end of the body (or any continue statement) copies the current value of the per-iteration i back to the per-loop i .
Unless a variable is captured like in the above example, nothing changes about how the loop executes. Adjusting the 3-clause form may seem strange to C programmers, but the same capture problems that happen in range loops also happen in three-clause for loops. Changing both forms eliminates that bug from the entire language, not just one place, and it keeps the loops consistent in their variable semantics. That consistency means that if you change a loop from using range to using a 3-clause form or vice versa, you only have to think about whether the iteration visits the same items, not whether a subtle change in variable semantics will break your code.
It is also worth noting that JavaScript is using per-iteration semantics for 3-clause for loops using let, with no problems. I think the semantics are a smaller issue than the idea of making this one-time gradual breaking change. I’ve posted this discussion to gather early feedback on the idea of making a change here at all, because that’s something we’ve previously treated as off the table. I’ve outlined the reasons I believe this case merits an exception below.
I’m hoping this discussion can surface concerns, good ideas, and other feedback about the idea of making the change at all (not as much the semantics). I know that C# 5 made this change as well, but I’ve been unable to find any retrospectives about how it was rolled out or how it went. If anyone knows more about how the C# transition went or has links to that information, please post that too. Thanks! The case for making the change: A decade of experience shows the cost of the current semantics I talked at Gophercon once about how we need agreement about the existence of a problem before we move on to solutions.
When we examined this issue in the run up to Go 1, it did not seem like enough of a problem. The general consensus was that it was annoying but not worth changing. Since then, I suspect every Go programmer in the world has made this mistake in one program or another. I certainly have done it repeatedly over the past decade, despite being the one who argued for the current semantics and then implemented them. (Sorry!) The current cures for this problem are worse than the disease.
I ran a program to process the git logs of the top 14k modules, from about 12k git repos and looked for commits with diff hunks that were entirely “x := x” lines being added. I found about 600 such commits. On close inspection, approximately half of the changes were unnecessary, done probably either at the insistence of inaccurate static analysis, confusion about the semantics, or an abundance of caution. Perhaps the most striking was this pair of changes from different projects: for _, informer := range c.informerMap { + informer := informer go informer.Run(stopCh) } for _, a := range alarms { + a := a go a.Monitor(b) } One of these two changes is unnecessary and the other is a real bug fix, but you can’t tell which is which without more context. (In one, the loop variable is an interface value, and copying it has no effect; in the other, the loop variable is a struct, and the method takes a pointer receiver, so copying it ensures that the receiver is a different pointer on each iteration.) And then there are changes like this one, which is unnecessary regardless of context (there is n
The rest of this page is on the live site.