[bjoli]

Yet, iterating over several collections and accumulating values is awkward.

It is not that it is easy to implement the iteration protocol, it is that the construct for looping universally sucks. Even the awful loop macro from common lisp runs laps around most other facilities. Racket's is even less powerful, but is miles better than what python offers.

Ruby can do some destructuring, but not enough to actually be useful.

[tangus]

It may be somewhat awkward; zipping collections to iterate over them in unison makes the first one (the sender) seem special, while it not necessarily is. And accumulating more than one value isn't always straightforward; you usually have to reduce explicitly. But it's so much better in the general case!

Like the example you showcase your macro with. In Ruby it would be:

    lst = [[1, 2], :dud, [3, 4], [5, 6]]
    lst.grep(Array).flatten.sum
    => 21

You can see what it does at a glance, you don't have to immerse yourself inside the directives to follow the logic. And when you can't do that, declaring iterators and accumulators outside a loop and then iterating and accumulating "by hand" yields code not so different than your more complex examples.

  def partition (arr)
    yes = []
    no = []
    arr.each do |elt|
      if yield elt
        yes << elt
      else
        no << elt
      end
    end
    [yes, no]
  end

[bjoli]

Your example of the flatten and grep will build two intermediate arrays before summing. That's inefficient.

I would also reach for higher order functions for most things, but when you need things to be fast that is not how you would do things. You would express it with nested for loops and ifs, and you would have to juggle state all by yourself. That is my main critique of looping constructs: they only cover the absolute simplest case, and then you are by yourself.

The partition example is not a particularly nice implementation of partition. It is there to show that you can write non-tail recursive loops, meaning you can use it to do tree walking and such. Using mutation, like your example, would make it prettier, but would make the code unsafe with schemes call/cc. Even a traditional named let (tail recursion) would be prettier.

[tangus]

For the record, if you want to avoid the creation of intermediate arrays, you can:

    lst.lazy.grep(Array).flat_map(&:itself).sum

Not as clear, because the standard library doesn't have a `#flatten` method for lazy enumerators.

But the point is the interface, not the implementation. Efficiency doesn't mandate assembly-like DSLs. Your interface could be as clear and clean as Ruby's and produce fast, inlined code by the power of macros. Ruby doesn't have macros, so chaining lambdas is the best it can do.

Ruby also has call/cc. None of the iterating methods has any provision to make them "safe" from it. They aren't safe from modifications to the iterated collection either. I think it makes sense; being forced to accumulate using only linked lists and having to be constantly on guard to support a rarely used quirky feature is a bad tradeoff IMO.

[bjoli]

I dont think interfaces is the point. As it is now in ruby you have to pick between comfort and speed when you can have both.

Continuation safety has never been a thing in ruby, so caring about it doesn't make much sense regardless of the presence of call/cc.

And lastly, neither my loops nor others I have mentioned only accumulate into lists. My loops a re in the most general sense a left fold, although right folds are possible to write as well. I provide accumulators for just about any data type available, and writing a new one yourself is pretty easy. You can express lazy loops with it as well.

All without sacrificing performance. Any performance in most cases. I haven't written much ruby since the 1.9 days, but back then I remember having to rewrite the nice .each{}.blah.blah into for loops. I could prototype in a nice way which was nice, but in the code I ended up shipping I had to juggle state using for loops because the thing I prototyped ended up being a bit slow.

I use map, filter and friends in scheme all the time, but when those are insufficient or too slow I don't have to resort to named lets. I can just use the loop macro which compiles to a named let.