You get this for free in Haskell, and you also save on not having to remember useless terminology for something that has no application on their own outside Foldables anyways.
It goes beyond a foldable, can be applied to streams. Clojure had foldables, called reducers, this was generalized further when core.async came along - transducers can be attached to core async channels and also used in places where reducers were used. The terminology is used to document the thing that various contexts accept (chan, into, sequence, eduction etc). They exist to make the language simpler and more general. They could actually allow a bunch of old constructs to be dispensed with but came along too late to build the whole language around.
> It goes beyond a foldable, can be applied to streams.
> Clojure had foldables, called reducers, this was generalized further when core.async came along - transducers can be attached to core async channels and also used in places where reducers were used.
Ok, you mean there's a distinction between foldables and the effectful and/or infinite streams, so there's natural divide between them in terms of interfaces such as (for instance) `Foldable f` and `Stream f e` where `e` is the effect context. It's a fair distinction, however, I guess my overall point is that they all have applicability within the same kind of folding algorithms that don't need a separate notion of "a composing object that's called a transducer" if you hop your Clojure practice onto Haskell runtime where transformations are lazy by default.
>...you also save on not having to remember useless terminology...
It may be true in this particular case, but in my admittedly brief experience using Haskell you absolutely end up having to remember a hell of a lot of useless terminology for incredibly trivial things.
Terminology doesn't bother me nearly as much as people defining custom operators.
I used to think it was cute the you could make custom operators in Haskell but as I've worked more with the language, I wish the community would just accept that "words" are actually a pretty useful tool.
Oh, my favorite part of the orange site, that's why we come here, that's the 'meat of HN' - language tribalism with a technical veneer. Congratulations, not only you said something as lame as: "French doesn't need the subjunctive mood because German has word order rules that already express uncertainty", but you're also incorrect factually.
Haskell's laziness gives you fusion-like memory behavior on lists for free. But transducers solve a broader problem - portable, composable, context-independent transformations over arbitrary reducing processes - and that you don't get for free in Haskell either.
Transducers exist because Clojure is strict, has a rich collection library, and needed a composable abstraction over reducing processes that works uniformly across collections, channels, streams, and anything else that can be expressed as a step function. They're a solution to a specific problem in a specific context.
Haskell's laziness exists because the language chose non-strict semantics as a foundational design decision, with entirely different consequences - both positive (fusion, elegant expression of infinite structures) and negative (space leaks, reasoning difficulty about resource usage).
> Haskell's laziness gives you fusion-like memory behavior on lists for free.
Haskell laziness & fusion isn't limited to lists, you can fuse any lawful composition of functions applied over data with the required lawful instances used for the said composition. There's no difference to what transducers are designed for.
> But transducers solve a broader problem - portable, composable, context-independent transformations over arbitrary reducing processes - and that you don't get for free in Haskell either.
Transducers don't solve a broader problem, it's the same problem of reducing complexities of your algorithims by eliminating transient data representations. If you think otherwise, I invite you to provide a practical example of the broader scope, especially the part about "context-independent transformations" that would be different to what Haskell provides you without that separate notion.
> and negative (space leaks, reasoning difficulty about resource usage).
which is mostly FUD spread by internet crowd who don't know the basics of call-by-need semantics, such as the places you don't bind your intermediate evaluations at, and what language constructs implicitly force evaluations for you.
> you can fuse any lawful composition of functions
each of those requires manually written rewrite rules or specific library support. It's not a universal property that falls out of laziness - it's careful engineering per data type. Transducers work over any reducing function by construction, not by optimization rules that may or may not fire.
> it's the same problem
It is not. Take a transducer like `(comp (filter odd?) (map inc) (take 5))`. You can apply this to a vector, a lazy seq, a core.async channel, or a custom step function you wrote five minutes ago. The transformation is defined once, independent of source and destination. In Haskell, fusing over a list is one thing. Applying that same composed transformation to a conduit, a streaming pipeline, an io-streams source, and a pure fold requires different code or different typeclass machinery for each. You can absolutely build this abstraction in Haskell (the foldl library gets close), but it's not free - it's a library with design choices, just like transducers are.
You're third claim is basically the "skill issue" defense. Two Haskell Simons - Marlow, and Jones, and also Edward Kmett have all written and spoken about the difficulty of reasoning about space behavior in lazy Haskell. If the people who build the compiler and its core libraries acknowledge it as a real trade-off, dismissing it as FUD from people who "don't know the basics" is not an argument. It's gatekeeping.
Come on, how can you fail to see the difference between: "Haskell can express similar things" with "Haskell gives you this for free"?
Why do you eliminate a library-based solution from the equation if it can actually prove the point that there's no difference in intent as long as my runtime is already lazy by default?
> It is not. Take a transducer like `(comp (filter odd?) (map inc) (take 5))`. You can apply this to a vector, a lazy seq, a core.async channel, or a custom step function you wrote five minutes ago. In Haskell, fusing over a list is one thing. Applying that same composed transformation to a conduit, a streaming pipeline, an io-streams source, and a pure fold requires different code or different typeclass machinery for each.
You can do that only because Clojure doesn't care whether the underlying iterable is to be processed by a side-effectful evaluation. That doesn't negate the fact that the underlying evaluation has a useless notion of "transducer". I said "fuse" in my previous comment to demonstrate that further comptime optimisations are possible that eliminate some transient steps altogether. If you don't need that you can just rely on generic lazy composition of functions that you define once over type classes' constraints.
`IsList` + `OverloadedLists` already exist. Had Haskell had a single type class for all iterable implicitly side-effectful data, you would have got the same singly-written algorithm without a single notion of a transducer. Let that sink in: it's not the transducer that's useful, it the differentiation between pure and side-effectful evaluations that allow your compiler to perform even better optimisations with out-of-order evaluations of pure stuff, as well as eliminating parts of inner steps within the composed step function, as opposed to focusing just on the reducing step-function during the composition. It's not a useful abstraction to have if you care about better precision and advanced optimisations coming from the ability to distinguish pure stuff from non-pure stuff.
Haskell aside, if your goal is to just compose reusable algorithms, a call-by-need runtime + currying + pointfree notation get you covered, you don't need a notion of transducers that exist on their own (outside of the notion of foldable interfaces) to be able to claim exactly the same benefits.
> Two Haskell Simons - Marlow, and Jones, and also Edward Kmett have all written and spoken about the difficulty of reasoning about space behavior in lazy Haskell.
There's a difference between what the people said in the past, and the things the crowd claims the people meant about laziness and space leaks. We can go over individual statements and see if they hold the same "negative" meaning that you say is there.
On IsList + OverloadedLists - this is a fantasy counterargument. Unified typeclass for side-effectful iterables doesn't exist in Haskell, so you're saying "had it existed, you'd get the same thing", you're describing a different language.
Transducers don't exist despite the lack of purity distinction, they exist because the reducing step function abstraction is useful regardless. You've drifted from "you get this for free" to "a different design with different trade-offs would make this unnecessary" - which again is just describing a different language. You're moving goalposts from "you get this for free" to "you don't need this at all if you design your language differently..."
Look, there are nice things in Haskell for sure, there are things that may cause frustration as well. Same for Clojure, but comparing them on a single thing is like judging a bicycle and a boat by which one flies better. They're built on fundamentally different assumptions and those assumptions cascade into every design decision. Transducers aren't a workaround for the absence of laziness, they're a natural solution within Clojure's actual constraints and goals - you're complaining without even understanding those constraints (in both of them). Haskell's laziness isn't a superior version of transducers, it's a different bet on different trade-offs. Neither language is trying to be the other.
Stick to Haskell if you must, bring to the table some interesting ideas from it, they'd be appreciated, but please stop spreading confusion and misinformation, thinking that if you talk louder people would prefer Haskell. It's not like folks en masse trying to abandon Python, Typescript and Java and confused between choosing Clojure or Haskell.
> so you're saying "had it existed, you'd get the same thing", you're describing a different language.
that's not what I'm saying. I'm saying that Haskell doesn't have it because it's a useless and shallow abstraction to have, that also hampers the ability to apply advanced optimisation laws down the compilation pipeline.
I will just repost the part that you conveniently ignored in your reply and pretended that it didn't exist:
Let that sink in: it's not the transducer that's useful, it the differentiation between pure and side-effectful evaluations that allow your compiler to perform even better optimisations with out-of-order evaluations of pure stuff, as well as eliminating parts of inner steps within the composed step function, as opposed to focusing just on the reducing step-function during the composition. It's not a useful abstraction to have if you care about better precision and advanced optimisations coming from the ability to distinguish pure stuff from non-pure stuff.
My argument holds: you get the same composability with lazy functions for free, you don't need to apply rewrite rules to be on the same level of reusability. Haskell grants you that for free, but for some reason you chime in and claim that's not the case and the only proof you've provided had to do with missing interfaces that can be solved by a library implementation. There's no restriction in the type system, nor runtime, to have it. But people don't need it because it's a useless abstraction that doesn't improve the baseline of what Haskell has to offer both in terms of composability of your foldings and further optimisations that take iteration purity into account.
> You've drifted from
I didn't drift from anything, I told you that you ignored a library-based solution in a sneaky attempt to move the goalpost from "you need rewrite rules in many places" to "there's no interface generic enough to accomodate effectful and non-effectful steps together without a library implementation".
> Haskell's laziness isn't a superior version of transducers
It absolutely is a superior solution to the same problem of algorithm optimisation and composability. It's more generic, it applies to anamorphisms and hylomorphisms in the same way as it does to foldings, and it doesn't introduce a special terminology to a single building block that doesn't exist outside foldings anyways.
> but please stop spreading confusion and misinformation
that's a bold statement coming from someone that claims that call-by-need semantics in Haskell is a negative aspect of the language according to other people (who probably didn't mean it in the first place, but you wouldn't dare to verify).