[instig007]

Why do you eliminate a library-based solution from the equation if it can actually prove the point that there's no difference in intent as long as my runtime is already lazy by default?

> It is not. Take a transducer like `(comp (filter odd?) (map inc) (take 5))`. You can apply this to a vector, a lazy seq, a core.async channel, or a custom step function you wrote five minutes ago. In Haskell, fusing over a list is one thing. Applying that same composed transformation to a conduit, a streaming pipeline, an io-streams source, and a pure fold requires different code or different typeclass machinery for each.

You can do that only because Clojure doesn't care whether the underlying iterable is to be processed by a side-effectful evaluation. That doesn't negate the fact that the underlying evaluation has a useless notion of "transducer". I said "fuse" in my previous comment to demonstrate that further comptime optimisations are possible that eliminate some transient steps altogether. If you don't need that you can just rely on generic lazy composition of functions that you define once over type classes' constraints.

`IsList` + `OverloadedLists` already exist. Had Haskell had a single type class for all iterable implicitly side-effectful data, you would have got the same singly-written algorithm without a single notion of a transducer. Let that sink in: it's not the transducer that's useful, it the differentiation between pure and side-effectful evaluations that allow your compiler to perform even better optimisations with out-of-order evaluations of pure stuff, as well as eliminating parts of inner steps within the composed step function, as opposed to focusing just on the reducing step-function during the composition. It's not a useful abstraction to have if you care about better precision and advanced optimisations coming from the ability to distinguish pure stuff from non-pure stuff.

Haskell aside, if your goal is to just compose reusable algorithms, a call-by-need runtime + currying + pointfree notation get you covered, you don't need a notion of transducers that exist on their own (outside of the notion of foldable interfaces) to be able to claim exactly the same benefits.

> Two Haskell Simons - Marlow, and Jones, and also Edward Kmett have all written and spoken about the difficulty of reasoning about space behavior in lazy Haskell.

There's a difference between what the people said in the past, and the things the crowd claims the people meant about laziness and space leaks. We can go over individual statements and see if they hold the same "negative" meaning that you say is there.

[iLemming]

On IsList + OverloadedLists - this is a fantasy counterargument. Unified typeclass for side-effectful iterables doesn't exist in Haskell, so you're saying "had it existed, you'd get the same thing", you're describing a different language.

Transducers don't exist despite the lack of purity distinction, they exist because the reducing step function abstraction is useful regardless. You've drifted from "you get this for free" to "a different design with different trade-offs would make this unnecessary" - which again is just describing a different language. You're moving goalposts from "you get this for free" to "you don't need this at all if you design your language differently..."

Look, there are nice things in Haskell for sure, there are things that may cause frustration as well. Same for Clojure, but comparing them on a single thing is like judging a bicycle and a boat by which one flies better. They're built on fundamentally different assumptions and those assumptions cascade into every design decision. Transducers aren't a workaround for the absence of laziness, they're a natural solution within Clojure's actual constraints and goals - you're complaining without even understanding those constraints (in both of them). Haskell's laziness isn't a superior version of transducers, it's a different bet on different trade-offs. Neither language is trying to be the other.

Stick to Haskell if you must, bring to the table some interesting ideas from it, they'd be appreciated, but please stop spreading confusion and misinformation, thinking that if you talk louder people would prefer Haskell. It's not like folks en masse trying to abandon Python, Typescript and Java and confused between choosing Clojure or Haskell.

[instig007]

> so you're saying "had it existed, you'd get the same thing", you're describing a different language.

that's not what I'm saying. I'm saying that Haskell doesn't have it because it's a useless and shallow abstraction to have, that also hampers the ability to apply advanced optimisation laws down the compilation pipeline.

I will just repost the part that you conveniently ignored in your reply and pretended that it didn't exist:

Let that sink in: it's not the transducer that's useful, it the differentiation between pure and side-effectful evaluations that allow your compiler to perform even better optimisations with out-of-order evaluations of pure stuff, as well as eliminating parts of inner steps within the composed step function, as opposed to focusing just on the reducing step-function during the composition. It's not a useful abstraction to have if you care about better precision and advanced optimisations coming from the ability to distinguish pure stuff from non-pure stuff.

My argument holds: you get the same composability with lazy functions for free, you don't need to apply rewrite rules to be on the same level of reusability. Haskell grants you that for free, but for some reason you chime in and claim that's not the case and the only proof you've provided had to do with missing interfaces that can be solved by a library implementation. There's no restriction in the type system, nor runtime, to have it. But people don't need it because it's a useless abstraction that doesn't improve the baseline of what Haskell has to offer both in terms of composability of your foldings and further optimisations that take iteration purity into account.

> You've drifted from

I didn't drift from anything, I told you that you ignored a library-based solution in a sneaky attempt to move the goalpost from "you need rewrite rules in many places" to "there's no interface generic enough to accomodate effectful and non-effectful steps together without a library implementation".

> Haskell's laziness isn't a superior version of transducers

It absolutely is a superior solution to the same problem of algorithm optimisation and composability. It's more generic, it applies to anamorphisms and hylomorphisms in the same way as it does to foldings, and it doesn't introduce a special terminology to a single building block that doesn't exist outside foldings anyways.

> but please stop spreading confusion and misinformation

that's a bold statement coming from someone that claims that call-by-need semantics in Haskell is a negative aspect of the language according to other people (who probably didn't mean it in the first place, but you wouldn't dare to verify).

[iLemming]

> It's more generic

Generic over what? Lazy evaluation is a semantic property of expression reduction. Transducers are parameterized over the reducing function. These aren't comparable on a generality axis - they live at different levels of abstraction. The fact that recursion schemes (ana/hylo) exist in Haskell is true and cool but doesn't address the actual transducer claim, which is: one value, applied to fundamentally different consumers (a channel, a fold, a stream, a transient collection) without recompilation or re-specialization. In Haskell, the closest analogs are conduit/pipes/streaming - each a library, each with its own type, each requiring adapters between them.

The concrete example - `(comp (filter odd?) (map inc) (take 5))` applied across source types - is the single most load-bearing thing in the thread and you never actually answered it. You gestured at OverloadedLists + a hypothetical unified typeclass, then pivoted to "it's useless anyway" which is the tell that you don't even understand the topic to start contemplating a direct answer.

Can we we please stop responding to concrete technical points by retreating to broader aesthetic claims - "useless", "shallow", "superior"? This honestly isn't helping anyone. I don't see the point of keeping going here, and not because I'm from the "internet crowd who don't know the basics".

You're claiming to know how (a better) language should have been designed, okay, let's talk about possibilities, instead of "just use Haskell" - that is really is childish.

[instig007]

> Generic over what?

Generic over whatever you decide to compose out of smaller parts into a full algorithm that doesn't produce transient buffered results. Transducers is a dead end of abstractions, they aren't applicable anywhere but folding, lazy runtime gets you covered for free regardless of your choice of the exact source of either a foldable `Stream f e a`, or a generator of values on demand, or even a data constructor.

> doesn't address the actual transducer claim, which is: one value, applied to fundamentally different consumers (a channel, a fold, a stream, a transient collection) without recompilation or re-specialization.

the transducer claim is that there's no way to track effects, period. Hey, you've found a new abstraction that doesn't care about things, my congratulations, you're now on par with Python itertools!

> In Haskell, the closest analogs are conduit/pipes/streaming - each a library, each with its own type, each requiring adapters between them.

Do you understand why it's the case? It's because transducers are useless and people actually care about further optimisations and experimentation. To be on par with Clojure it would be enough to have a single `Stream m e a` that everyone would silently buy into. But no one opts for it, because people actually care about their local optimisations that go beyond what you think transducers give you. If you don't care about those, pick any generic enough interface and glue it with whatever you want in a single place for the entirey of your ecosystem. Had `Streamly` been part of `base`, you'd get exactly that property that you claim isn't a thing. Then maybe add `streamly` into your dependency list and start using it pervasively everywhere where iteration happens. You'll be on par with Clojure, but without the silly notion of transducers as a thing of its own (but it's not, it's only for foldings that don't care about side-effects).

[iLemming]

> Transducers aren't applicable anywhere but folding

Wrong, factually wrong! Transducers apply to anything expressible as a step function: reductions, yes, but also channels (core.async), observable streams (manifold), eduction pipelines, into-transformations, transient-collection builds, stateful transformations like partition-by and dedupe that don't fit a pure fold at all. (dedupe) is a transducer. Try expressing it as a pure lazy-list fusion. You can, but you need explicit state threading, and then you've rebuilt a step function by hand.

The definition of "folding" you're using here is so broad it's doing no work. If "folding" means "any left-to-right consumption of values", then yes, transducers are for folding - and so is ~all of streaming, ~all of iteration, ~all of channel consumption. You're using the word to make the scope sound small while the scope is actually most of what programs do with sequences of values.

> the transducer claim is that there's no way to track effects, period.

The transducer claim - the actual one, as stated in Hickey's talk and the docs - is that a reducing function is a fundamental substrate that composes, and you can build transformations over reducing functions that are source and sink-agnostic. Effect tracking is orthogonal.

You keep trying to move the goalposts to "transducers must track effects or they're useless". That's like saying "typeclasses must handle concurrency or they're useless" It's a category demand imported from your preferred language's feature set.

The "on par with Python itertools" jab is wrong. itertools composes over iterables only. Transducers compose over reducing functions. Python itertools does not work against asyncio.Queue or a user-defined reduce.

> To be on par with Clojure it would be enough to have a single `Stream m e a` that everyone would silently buy into.

Okay, let me read that again slowly:

1. The property you're describing (one transformation, many consumers) is real and distinct.

2. Haskell does not currently give it to you (on the language level).

2. To give it to you, Haskell would need a single blessed streaming abstraction in base.

3. Haskell doesn't have one because the community prefers local optimization over a universal substrate.

The rationalization is fine - yes, there's a real trade-off between "single blessed abstraction for everyone" and "multiple specialized libs, each optimized for its niche" - but it is a trade-off. You've started with "you get this for free in Haskell", and arrived to: "Haskell correctly chose not to give you this, and here's why the thing you want is actually bad..."

Streamly is a great Haskell library, does streaming well, has effect tracking, is performant. And it is absolutely not a drop-in transducer analog - Streamly composes over Streamly streams. If you have a conduit source, a pipes producer, and a streaming Stream, Streamly doesn't make one composed transformation apply to all three. It just adds a fourth ecosystem. So your "had Streamly been in base" hypothetical is exactly the Clojure move - pick one substrate, bless it, get uniformity - and now you're simultaneously using Streamly to argue that Haskell doesn't need transducers while pointing at a hypothetical world where Haskell would have done what Clojure actually did. "pick any generic enough interface and glue it with whatever you want in a single place for the entirety of your ecosystem" - this is basically what Clojure did.

Can we just find a middle ground in this debate that maybe actually works, something like:

"Sure, Clojure blessed a universal reducing substrate at the language level. Haskell didn't, and instead has multiple streaming libraries, each with stronger local guarantees about effects, memory, and back-pressure. Clojure trades uniformity across effect context - a transducer, works everywhere, at the cost of the compiler not telling you whether a given pipeline touches the world; Haskell chose effect-visibility in types"

Neither side is free. Clojure pays in runtime-only knowledge of effects. Haskell pays in fragmentation of streaming abstractions and the attendant ceremony of moving between them. That's the trade. It's not flaws being papered over; it's the shape of the bet. You can argue the bet is wrong, but you can't argue it wasn't made on purpose.