[travisd]

It's worth noting that it's much less of a problem in Python due to the lack of ergonomic closures/lambdas. You have to construct rather esoteric looking code for it to be a problem.

    add_n = []
    for n in range(10):
        add_n.append(lambda x: x + n)
    add_n[9](10)  # 19
    add_n[0](10)  # 19

This isn't to say it's *not* a footgun (and it has bit me in Python before), but it's much worse in Go due to the idiomatic use of goroutines in a loop:

    for i := 0; i < 10; i++ {
        go func() { fmt.Printf("num: %d\n", i) }()
    }

[eru]

In Python you are much more likely to hit that problem not with closures constructed with an explicit 'lambda', but with generator-comprehension expressions.

    (((i, j) for i in "abc") for j in range(3))

The values of the above depends on in which order you evaluate the whole thing.

(Do take what I wrote with a grain of salt. Either the above is already a problem, or perhaps you also need to mix in list-comprehension expressions, too, to surface the bug.)

[nerdponx]

Yeah, this one is weird:

  gs1 = (((i, j) for i in "abc") for j in range(3))
  gs2 = [((i, j) for i in "abc") for j in range(3)]

  print(list(map(list, gs1)))
  print(list(map(list, gs2)))

Results:

  [[('a', 0), ('b', 0), ('c', 0)], [('a', 1), ('b', 1), ('c', 1)], [('a', 2), ('b', 2), ('c', 2)]]
  [[('a', 2), ('b', 2), ('c', 2)], [('a', 2), ('b', 2), ('c', 2)], [('a', 2), ('b', 2), ('c', 2)]]

That's a nice "wat" right there. I believe the explanation is that in gs2, the range() is iterated through immediately, so j is always set to 2 before you have a chance to access any of the inner generators. Whereas in gs1 the range() is still being iterated over as you access each inner generator, so when you access the first generator j=1, then j=2, etc.

Equivalents:

  def make_gs1():
      for j in range(2):
          yield ((i, j) for i in "abc")

  def make_gs2():
      gs = []
      for j in range(2):
          gs.append(((i, j) for i in "abc"))
      return gs

Late binding applies in both cases of course, but in the first case it doesn't matter, whereas in the latter case it matters.

I think early binding would produce the same result in both cases.

[_ZeD_]

or you could just be eager and use lists:

    >>> [[(i, j) for i in "abc"] for j in range(3)]
    [[('a', 0), ('b', 0), ('c', 0)], [('a', 1), ('b', 1), ('c', 1)], [('a', 2), ('b', 2), ('c', 2)]]

[nerdponx]

Right, creating generators in a loop is not usually something you want to do, but it's meant to demonstrate the complexity that arises from late binding rather than demonstrate something you would actually want to do in a real program.

[Spivak]

Ignoring the strange nature of this code in the first place the more pythonic way to do it would be

    from functools import partial
    from operators import add

    add_n = [partial(add, n)) for n in range(10)]

    assert add_n[5](4) == 9

Look ma, no closures.

[eru]

'partial' creates a closure for you.

[Spivak]

Unless you're talking philosophically how classes and closures are actually isomorphic then no, it doesn't. None of the variables in the outer scope are captured in the class instance.

https://github.com/python/cpython/blob/main/Lib/functools.py...

Here's a simplified version of that code that demonstrates the pattern.

    class partial:
      def __init__(self, func, *args, **kwargs):
        self.func = func
        self.args = args
        self.kwargs = kwargs

      def __call__(self, *args, **kwargs):
        return self.func(*self.args, *args, **(self.kwargs | kwargs))

      p = partial(add, 5)  # -> an instance of partial with self.args = (5,)
      res = p(4)  # -> calls __call__ which merges the args and calls add(5, 4)

[eru]

I was talking 'philosophically' in that sense. The partial object does create a new scope that binds a few of those variables.

But you are also right that the mechanisms in Python are different (on some suitable mid-level of abstraction) for those two.

[hinkley]

Everyone else solved this problem by using list comprehensions instead. Rob has surely heard of those.

[lmm]

Of the two comprehension syntaxes in Haskell, Python picked the wrong one. Do notation (or, equivalently, Scala-style for/yield) feels much more consistent and easy to use - in particular the clauses are in the same order as a regular for loop, rather than the middle-endian order used by list comprehensions.

[eru]

Haskell has both do-notation and list comprehension.

Comprehension in both Python and Haskell (for both lists and other structures) use the same order in both language, as far as I remember.

[lmm]

> Haskell has both do-notation and list comprehension.

Right, and do-notation is the one everyone uses, because it's better. Python picked the wrong one.

> Comprehension in both Python and Haskell (for both lists and other structures) use the same order in both language, as far as I remember.

It may be the same order as Haskell but it's a terrible confusing order. In particular if you want to go from a nested list comprehension to a flat one (or vice versa) then you have to completely rearrange the order it's written in, whereas if you go from nested do-blocks to flat do-blocks then it all makes sense.

[eru]

I see what you mean, but I don't find the order that confusing in neither Haskell or Python.

However, I can imagine a feature that we could add to Python to fix this: make it possible for statements to have a value. Perhaps something like this:

    my_generator = \
      for i in "abc":
        for b in range(3):
          print("foo")
          yield (i, b)

or perhaps have the last statement in block be its value (just like Rust or Ruby or Haskell do with the last statement in a block), and make the value of a for-loop be a generator of the individual values:

    my_list = list(
      for i in "abc":
        for b in range(3):
          (i, b))

Though there's a bit of confusion here whether the latter examples should be a flat structure or a nested one. You could probably use a similiar mechanism as the existing 'yield from' to explicitly ask for the flat version, and otherwise get the nested one:

    my_list = list(
      for i in "abc":
        yield from for b in range(3):
          (i, b))

Making Python statements have values looks to me like the more generally useful change than tweaking comprehensions. You'd probably not need comprehension at all in that case. Especially since you can already write loop header and body on a single line like

    for i in range(10): print(i)

if they are short enough.

[nerdponx]

  for i in range(10): print(i)

But what would that return? [None] * 10?

The limited whitespace-based syntax limits the potential for fun inline statement things, but it also completely dodges the question of what any particular statement should evaluate to when used as an expression.

[camgunz]

When I was starting in Python years ago I had to turn my brain inside out to learn how to write list comprehensions. Sometimes I wonder what it's like to be a normal person with a normal non-programmer brain, having forgotten it entirely these last many years.

[nerdponx]

But Python doesn't have any concept of a monad, so what would do-notation even be in Python? And who is the "everyone" using do-notation? I don't see any analogous syntax in Lua, Javascript, Ruby, or Perl.

In Python there is a nice tower of abstractions for iteration, but nothing more general than that, so it makes perfect sense IMO to use the syntax that directly evokes iteration.

The existing syntax is meant to mirror the syntax of a nested for loop. I agree that maybe it's confusing, but if you want to go from a multi-for comprehension to an actual nested for loop, then you don't have to invert the order.

[lmm]

> But Python doesn't have any concept of a monad, so what would do-notation even be in Python?

It could work on the same things that Python's current list comprehensions work on. I'm just suggesting a different syntax. Comprehensions in Haskell originally worked for all monads too.

> And who is the "everyone" using do-notation? I don't see any analogous syntax in Lua, Javascript, Ruby, or Perl.

I meant that within Haskell, everyone uses do notation rather than comprehensions.

> The existing syntax is meant to mirror the syntax of a nested for loop. I agree that maybe it's confusing, but if you want to go from a multi-for comprehension to an actual nested for loop, then you don't have to invert the order.

You have to invert half of it, which I find more confusing than having to completely invert it. do-notation style syntax (e.g. Scala-style for/yield) would keep the order completely aligned.

[rofrol]

https://wiki.haskell.org/Do_notation_considered_harmful

[panzi]

How does list comprehension change anything here? This has the same problem:

    add_n = [lambda x: x + n for n in range(10)]
    add_n[9](10)  # 19
    add_n[0](10)  # 19

[billyjmc]

I’m not sure what they mean by list comprehensions, either, but for completeness’s sake, I must point out that this is solvable by adding `n` as a keyword argument defaulting to `n`:

    add_n = [lambda x, n=n: x + n for n in range(10)]
    add_n[9](10)  # 19
    add_n[0](10)  # 10

[drekipus]

This is the way

Also pylsp warns you about this

[simiones]

I don't think anyone is puzzled by the Go snippet being wrong.

The bigger problem in Go is the for with range loop:

  pointersToV := make([]*val, len(values))
  for i, v := range values {
    go func() { fmt.Printf("num: %v\n", v) } () //race condition
    pointersToV[i] = &v //will contain len(values) copies of a pointer to the last item in values
  }

This is the one they are changing.

Edit: it looks like they're actually changing both of these, which is more unexpected to me. I think the C# behavior makes more sense, where only the foreach loop has a new binding of the variable in each iteration, but the normal for loop has a single lifetime.

[o11c]

It's actually worse in Python since there's no support for variable lifetimes within a function, so the `v2` workaround is still broken. (the default-argument workaround "works" but is scary)

This makes it clear: the underlying problem is NOT about for loops - it's closures that are broken.

[nerdponx]

It's not broken, it's a different design. Maybe worse in a lot of cases, but it's not broken. It's working as intended.

[eru]

You could say the design is broken, but the implementation is working as intended by the design.

[jshen]

Doesn’t go vet complain about your code? I’m not at my computer right now so can’t check.

[_ikke_]

> Tools have been written to identify these mistakes, but it is hard to analyze whether references to a variable outlive its iteration or not. These tools must choose between false negatives and false positives. The loopclosure analyzer used by go vet and gopls opts for false negatives, only reporting when it is sure there is a problem but missing others.

So it will warn in certain situations, but not all of them

[simiones]

Why would it? It's perfectly correct code, it's just not doing what you'd expect.

It might complain about the race condition, to be fair, but the same issue can be reproduced without goroutines and it would be completely correct code per the semantics.

[jshen]

In many languages "if x = 3" is perfectly valid code, but almost certainly not what the person intended "if x == 3". It's very smart to warn someone in a scenario like this.

[simiones]

I don't really write C too much, but I thought `if err = functionWithErrorReturn() { handleError(err) }` was a somewhat common idiom.

[defrost]

It's a common enough idiom from "stone age" bare bones K&R C, absolutely.

It's also one of the great foot-guns of C programming as there are so many other almost but not that idioms and it's never clear on casual inspection whether the result of an assignment was meant to be examined or the result of a comparison.

With the evolution of C and C sanity tools that rightfully flag such statements for double checking and the desire to not have spurious flagging, etc. it's more common in later C code to see (say)

    if ((err = someFunction()) != NOERROR) { errorHandle(err) }

that optimises down to the same intermediate code where NOERROR is 0, sure, but it makes it very clear what is going on, an intended assignment and then an intended comparison.

As with all idoms the general practice in the larger codebase and house code standard rules apply - there are other ways of doing similar things.

[Frotag]

I've run into this once. IIRC the workaround was to add a n=n arg to the lambda