Hacker News new | ask | show | jobs
by samwillis 1266 days ago
Map/filter are considered inferior in Python to list comprehensions.

  res = [x**2 for x in range(10) if x != 5]
4 comments

Before Ruby introduced filter_map:

res = (1..10).select { |x| x != 5 }.map { |x| x ** 2 }

With filter_map:

res = (1..10).filter_map { |x| x ** 2 if x != 5 }

In both cases, I think the Ruby solution is more readable.

Python list comprehensions invert the subject (data) and the verb (action). You see what will be done before you see what the subject is. I would argue that showing the subject first allows easier code review as you know immediately what you are working with.

But beyond that, the first Ruby example tells you in English what is happening. "take this range", "select a subset", then "map some actions to the elements".

And the filter_map abbreviation does the same, telling you "take this range, filter it and perform an operation on the remaining elements".

Python tells you nothing... and what it does say is in awkward order.

As functional and data-oriented programming is gaining in popularity (for good reason), adopting some functional practices in Ruby is a pleasant experience. Doing the same in Python exposes more of these... irregularities.

Edit - I always forget how to format symbols in these comments!

The Python one looks fine to me, although I am a Pythonish person.

It uses what people already know: the for something in somethings syntax of the for loop, and the if syntax. Also it's nice that this works in dictionaries, generators and lists.

It also has the same narrative flow of Haskell's list comprehensions, which I think come from set theory:

  [x^2 | x <- [0..10], x `mod` 5 /= 0]
As for your Ruby examples: I think you could argue that the filter_map version is very readable, but not necessarily more so, but the select one looks pretty painful.
> As for your Ruby examples: I think you could argue that the filter_map version is very readable, but not necessarily more so, but the select one looks pretty painful.

The select does two passes, which makes it quite inefficient. One does not even need filter_map, since the example is essentially a reduce operation.

   res = (1..10).reduce([]) { |a, x| x != 5 ? a.push(x**2) : a }
This works in ruby 2.5.1. Probably works in 1.9 and mruby as well.
True, although that seems less readable than the comprehension versions. I might be just biased, though.
you can define pretty much everything as a reduce operation though
> I think the Ruby solution is more readable.

I disagree and I’ve used both professionally for about the same amount of code.

I think this is purely a personal preference but I also think there is a bias towards list comprehensions being more difficult to mentally parse.

I do a lot of contract work and chatted with a ton of folks ranging from beginners to veterans. A lot of them (well more than half) avoid list compressions, especially when working with teams because it's such a mixed bag of either being able to instantly understand them or it requires more effort. Personally I don't use them in my code (for both reasons).

Both Ruby solutions are much more clear to me even though I have no functional programming background. I have no preference towards functional styles either, I would say it's the opposite. I struggled with Elixir long enough that I stopped using it.

I disagree about list comprehensions, as do more than half the people I’ve asked over the years.
I don’t use either, and also have an admittedly irrational dislike for Python. That said, the Python variant is more readable imo as well.
And yet, in the production Python codebases I've worked with, list comprehensions are rarely seen. Usually it's typical loop iterations. I wonder why that is?...
This says much more about you and the developers and codebases you work with than anything to do with Python list comprehensions.

Coming from a point of zero knowledge of the codebase, I picked Flask. I picked the cli.py module in Flask. And what do I find?

https://github.com/pallets/flask/blob/main/src/flask/cli.py#...

https://github.com/pallets/flask/blob/main/src/flask/cli.py#...

https://github.com/pallets/flask/blob/main/src/flask/cli.py#...

https://github.com/pallets/flask/blob/main/src/flask/cli.py#...

https://github.com/pallets/flask/blob/main/src/flask/cli.py#...

https://github.com/pallets/flask/blob/main/src/flask/cli.py#...

https://github.com/pallets/flask/blob/main/src/flask/cli.py#...

https://github.com/pallets/flask/blob/main/src/flask/cli.py#...

The keyword "for" occurs twice as often in comprehensions and generator expressions as it does in "typical loop iterations!"

Thinking further about this, how does this work with multiple loops? E.g.

  [x + y for x in range(10) for y in range(5)]
This may be a nitpick but Ruby’s naming seems inconsistent. If `filter_map` combines `select` and `map`, why is it not called `select_map`?
> In both cases, I think the Ruby solution is more readable.

Nope.

By who?

I always have to stare at list comprehensions very closely to understand operation being done. The source of the data is in the middle, where it should logically come first. The filter is at the end, where logically it should come after the source. The mapping is at the start, where logically it should come at the end.

I find the monadic, additive style of Ruby much easier to understand:

    10.times.select { |x| x != 5 }.map { |x| x ** 2 }
IMO it's more composable. What if you want to exclude even numbers from the result? Just add another filter:

    10.times.select { |x| x != 5 }.map { |x| x ** 2 }.select { |x| x % 2 != 0 }
Incrementally building up a streaming computation this way is much more useful to me than a list comprehension.

For example, you can add a lazy to the stream to avoid performing all the operations eagerly, and now you have a way to process sequences without blowing out your memory.

> By who?

By the Python developers and its wider community. As Python doesn't have anonymous function blocks in the same way as Ruby (only lambda expressions), tutorials, lessons and the Python docs steer users toward list comprehensions instead.

I'm not saying the ruby syntax is not elegant (it is), I'm saying in Python list comprehensions are recommended over filter/map functions.

On the composable front, personally I prefer to breaks these down into smaller chunks with descriptive variable names rather than chaining.

Python also has the sister "generator" (() rather than []) syntax which also ensures it remains efficient as it pipelines the whole sequence of generators. (Lazily rather than eagerly as you say)

You're effectively saying something like: Canadians prefer Canada. What about the rest of the world?

Once you start adding more "and" to the if-statement in the list comprehension, it becomes a mess. Breaking them down to smaller chunks is required because comprehensions are messy. You are doing smaller chunks due to a shortcoming of comprehensions. Chaining is nice option to have, especially when the chained functions are straight forward.

> Map/filter are considered inferior in Python to list comprehensions.

> By who?

> By the Python developers and its wider community.

> You're effectively saying something like: Canadians prefer Canada. What about the rest of the world?

They're actually saying that Python developers prefer one particular way of doing something rather than a different particular way of doing the same thing. You're suggesting that they're saying Python developers (Canadians) prefer Python (Canada).

I don't mean to speak against your broader points, just that this specific call out is mistaken.

That's an utter mess of control flow right here. Read it left to right ? Wrong. Right to left ? wrong.

Looking at it makes me miss Perl oneliners

Map, filter and similar can be chained together and composed much better than comprehensions I think. I realize that it's probably not your opinion that comprehension are superior, (although it might be) but rather the general python style.

I also think that map and filter style computations can be much more powerful, there are quite a few things other than just map and filter, like count, take, skip, find, flatten, fold, map-while and quite a few more!

In python I guess you are supposed to use a standard for loop to do these things instead.

No. You are supposed to use intermediate variables that contain generators, and write short named functions to call within generator expressions, instead of writing big run-on chains and compositions with little bits of anonymous logic floating around in it.

The = binding is the chaining/composition tool of choice. This is why generator expressions are so important, relative to list and dict comprehensions. They both defer the allocation of space for intermediate values and allow the space to be bounded no matter what the input length is.

Often the “reduce” step, or even more commonly, realizing the side effects of such a generator composed of generators is a simple for loop—because that’s the most readable way to walk through it.