Hacker News new | ask | show | jobs
by thomasjames 2820 days ago
Isn't the property of the generator expression not that it is an iterator (like a range or list also are) but specifically that it is a generator and thus inherently has state that is permanently/mutably exhaustible across different scopes? I like the example, and this is the first time I've totally wrapped my head around Rust ownership, so thanks! I just think precise Python terminology might keep people from getting confused about different types of Python iterator objects. Iterators are anything that has a __next__() method: https://www.python.org/dev/peps/pep-0234/
2 comments
rahimnathwani 2820 days ago
"it is an iterator (like a range or list also are)"

No, range is not an iterator. Try this in either python 2 or 3:

  a = range(10)
  a.next()
It will fail. Now try this:

  a = range(10)
  my_iterator = a.__iter__()
  my_iterator.next()
"Iterators are anything that has a __next__() method"

Right, and range(10) does not have a __next__() method, so it's not an iterator. It's an iterable, which is anything which has a __iter__() method that returns an iterable.

In python 3, range(10) returns a range object (not a list). Because it's an iterable and not an iterator, it doesn't get 'used up'. For example, try this (in Python 3 only):

  a = range(1000000000) # returns instantly, as it's not building a list
  a[3] # returns 3
  a[3] # returns 3 again
link
deathanatos 2820 days ago
> Isn't the property of the generator expression not that it is an iterator (like a range or list also are) but specifically that it is a generator and thus inherently has state that is permanently/mutably exhaustible across different scopes?

> just think precise Python terminology might keep people from getting confused about different types of Python iterator objects. Iterators are anything that has a __next__() method

An iterator will function exactly as poorly as a generator does in his first example:

  In [5]: l = [1, 2, 3]

  In [6]: iter_ = iter(l)

  In [7]: min(iter_)
  Out[7]: 1

  In [8]: max(iter_)
  ---------------------------------------------------------------------------
  ValueError                                Traceback (most recent call last)
  <ipython-input-8-0dab0672ef83> in <module>()
  ----> 1 max(iter_)

  ValueError: max() arg is an empty sequence
(I chose iter(a list) here as it is not a generator.)

And for the same reasons, really. An Iterator in Python (something w/ a __next__ method) also keeps mutable state, and the first call to min() exhausts it (by mutating it). (In Rust, it would likely "consume" the iterator, that is, it would move it, and the article demonstrates an example of this.) The mutable state being kept and used for iteration is the material part here; the generator is just an easy way of obtaining an iterator.

Hopefully I've got this right, but in Python, all generators are iterators¹ (they have a __next__); generators also additionally have send(), throw(), and close() methods, making them substantially more powerful. They make it really easy to make just a basic iterator, so they're often used for that.

¹The glossary actually calls the functions (those with "yield") themselves generators, and the objects returned by them "generator iterators".

link