[gue5t]

I'm not sure you're characterizing Rust iterator adaptors right.

Iterator adaptors take an iterator (which usually, but not always, comes from a collection, e.g. "my_vec.into_iter()") and give you another iterator back, without allocating an intermediate collection; the iterator itself stores only the current state of the iteration process. You can apply iterator adaptors to such iterators as the receiving end of a channel ( http://doc.rust-lang.org/std/sync/mpsc/struct.Receiver.html#... ), and you can pass around the iterator and add other computations to the end of it before you "run" the whole thing.

However, I'm not sure what you mean by the transformation processes being functions. In Rust these processes (iterators) are values, and have methods that produce a derived process by appending a step; for a concrete example:

    let fb_chars = "foobar".chars(); //fb_chars is an iterator
    let not_o = fb_chars.filter(|&x| x!='o'); //not_o is an iterator; we got it by applying the filter iterator adaptor
    for i in not_o {println!("char: {}", i);} //run the iterator with a for loop; we could also have run it by calling .collect() and obtaining a collection

What are the domain and range of the functions you mention in Clojure?

[bad_user]

I guess Rust's iterators are pretty standard, meaning that you keep calling "next(): T" until there are no more elements, right?

Well that's pretty cool and iterators are generic, but not generic enough. If you'll look closely their protocol has certain constraints. For example the next() call is synchronous. If the next element is not ready, well, tough luck, the current thread needs to be blocked. Another constraint is that iterators produce elements, right? Well, iterators are cool and useful, but many operators like map(), filter() and flatMap() can be applied on things that don't necessarily produce values. As a side note, this is why monads are hard to understand, being a little more abstract than iterators.

What transducers from Clojure are doing is to define a more generic protocol (more generic than that of Iterator) such that you can have defined operators, like map and filter, operate on whatever you want, such that you can reuse the implementation of those operators. You can also compose those operators, before applying them to something concrete.

[Veedrac]

On further thought, the genericism of transducers seems to hurt when you want to combine multiple streams, even somewhat trivially:

   zip_flat xs ys = zip (flatten xs) (flatten ys)

I don't see how you could implement this as a transducer, since a transducer isn't able to reason about its inputs.

[Veedrac]

> Well, iterators are cool and useful, but many operators like map(), filter() and flatMap() can be applied on things that don't necessarily produce values.

You've lost me. You need to apply the function in `flatMap` to something, and `flatMap` by definition will output zero or more values.

So what do you mean?

[bad_user]

The signature of flatMap is something like this:

    flatMap[M,A,B](cc: M[A], f: A => M[B]): M[B]

So given that you respect its signature, M[A] is not necessarily a producer of values. It can be anything and it helps if you think about it not like a collection or a container, but more like a context. For example M[A] could be a function of type S => (A, S). And then you could map and flatMap on such functions to produce other functions. This is the "state monad" btw.

[llasram]

I could have been clearer. By the scare-quoted "collection" I meant a value which implements the relevant collection-like interface. In Clojure this is any object `reduce` works on, in Rust is any struct implementing the `Iterator` trait etc. In all the examples discussed these transformations are lazy, only execution the composition of any number of transforms when values are requested.

Clojure transducers are different because a transducer is a function which accepts a reducing function and returns a new reducing function, adding behavior by how the input function is composed into the result function. Because the domain and range of transducers are both reducing functions, they can be chained through function composition. A chain is actually applied by transforming a reducing function then using that function to `reduce` (Rust `fold`) a collection.

The cool part is that the transformations don't refer to collections at all, not even through some highly abstract collection-like interface. This makes them applicable to other domains, like the previously-mentioned channels.

[gue5t]

Defining a transducer involves returing a closure wrapping some logic around calling a function of type (result, element) → result to get a function of type (result', element) → result'. All the while you're composing, I assume you don't know what this element type is?

Defining an iterator adaptor involves defining the state itself (generally you have a field for the state of the iterator you wrap, and some fields for the state of your transformation itself) and its .next() method for the Iterator impl. The .next() method has type (state, input) → (state, optional output), except that in Rust reality it mutates the state rather than returning a new one.

The difference seems to be that in Rust the output is separate from the state. You could convert an iterator adaptor into a transducer by making the optional output a field in the state.

Also, in Clojure it looks like the usage is that you compose the transducers separately from applying to an input, whereas in Rust you generally compose adaptors on top of the input iterator. I assume this is what you mean with "not referring to the collection at all". In Rust, defining the adaptor chain ahead of applying it would look like:

    let transducer = |x: Iterator| {x.map(xfrm).filter(pred).whatever()};

Which is fine, but not particularly idiomatic, and might require some work annotating types.

[Veedrac]

> Because the domain and range of transducers are both reducing functions, they can be chained through function composition. A chain is actually applied by transforming a reducing function then using that function to `reduce` (Rust `fold`) a collection.

For iterators:

Because the domain and range of iterator adaptors are both iterators, they can be chained through function composition. A chain is actually applied by transforming an iterator adaptor then using that iterator adaptor to "iterate" a collection.