[lmm]

* Tracking which things need to happen in a database transaction

* Gathering statistics (multiple different cases)

* Authentication

* Async pipelines (I found iteratees much easier to learn than "reactive streams" because they're just monads)

Essentially any time you find yourself with a "cross-cutting concern", something you'd be tempted to use an "aspect" or "decorator" for, you probably want to use a monad. And there's a lot of complicated language features that you can just remove (or reduce to syntax sugar) if your language has monads instead: https://philipnilsson.github.io/Badness10k/escaping-hell-wit...

[ezrast]

Thanks, those are good examples. The linked article underscores my point though: the author takes four disparate ideas and reduces them to the exact same code in each case. All that code tells me is that there are monads going on; that's no good to me if I can't tell what those monads are actually doing!

Also, for what it's worth you can get rid of a lot of those same language features with good old OO methods. See Crystal's each[1] and try[2] methods, for example.

I don't mean this to be a tit-for-tat and do appreciate the response. You've inspired me to try and make use of a state monad in some Ocaml I've been playing with. Cheers.

[1]https://crystal-lang.org/api/0.35.1/Indexable.html#each(&)-i... [2]https://crystal-lang.org/api/0.35.1/Object.html#try(&)-insta...

[lmm]

> Thanks, those are good examples. The linked article underscores my point though: the author takes four disparate ideas and reduces them to the exact same code in each case. All that code tells me is that there are monads going on; that's no good to me if I can't tell what those monads are actually doing!

This is like saying that a generic list type makes no sense because you can't tell what the values in the list actually are. The whole point of using monads is so that you can separate the generic structure of what you're doing (composing together effectful functions) from the specific details of what those effects are, and there are a bunch of useful generic functions that you can write (analogy: sorting the list, or combining together two lists) and reuse for any kind of effect.

> Also, for what it's worth you can get rid of a lot of those same language features with good old OO methods. See Crystal's each[1] and try[2] methods, for example.

Sure. I think if you actually really follow through on OO principles like SOLID and separation of concerns then you end up in much the same place as functional programming. If you replace that "each" and "try" with versions that return a "transformed" value (more composable and testable than just executing a function for side effects) then you more-or-less have the "map" function.

(Unfortunately most languages aren't sophisticated enough to let you pull out that common interface: I'm sure you can see that a function List<Int> -> List<String> has something in common with a function Set<Int> -> Set<String> or a function Future<Int> -> Future<String>, but programming languages that let you express what that common part is are surprisingly rare).

[ezrast]

I'm not saying the type makes no sense, I'm saying the article doesn't communicate any value to people who aren't sold on the idea of monads before they go in. Like, if you're explaining functions to a newbie programmer, you don't just take a bunch of lines of code, replace them with `do_the_function()`, and act like you've made the program better. Obviously all those lines are still hiding in the function definition somewhere and your indirection has only increased complexity. You have to show that using functions to extract patterns from around the program improves elegance or expressiveness overall (e.g. by reducing total lines of code). I've never read a monad tutorial that does this in a way that (comparatively) simple higher-order functions wouldn't do just as well.

edit: to expand on that last sentence, the example under "Ad hoc solution: Promises" is more expressive to me than the author's monadic code, because a) it's more explicit about what depends on what, and b) the ".then" method name gives you a hint about what sort of computation is actually going on. Replacing "then" with "try" or "each" makes the code work just as well for the null-checking and for-loop examples.

[lmm]

> I'm not saying the type makes no sense, I'm saying the article doesn't communicate any value to people who aren't sold on the idea of monads before they go in. Like, if you're explaining functions to a newbie programmer, you don't just take a bunch of lines of code, replace them with `do_the_function()`, and act like you've made the program better. Obviously all those lines are still hiding in the function definition somewhere and your indirection has only increased complexity.

I'm not convinced. Imagine a tutorial for the foreach construct: it would show code for iterating over a list, code for iterating over a set, and code for iterating over an array, and then a foreach loop. Would you say that obviously the iteration code is still hiding somewhere and your indirection has only increased complexity? Introducing a common interface - which is proven by the use of a common syntax - is something that we recognise as valuable, I think.

> You have to show that using functions to extract patterns from around the program improves elegance or expressiveness overall (e.g. by reducing total lines of code). I've never read a monad tutorial that does this in a way that (comparatively) simple higher-order functions wouldn't do just as well.

FWIW my own effort is https://m50d.github.io/2013/01/16/generic-contexts

[ezrast]

An interface is a kind of relation between things that would otherwise be separate. So yes, I'd say that establishing the interface by itself increases complexity slightly.

However, there are lots of situations where one may want to substitute one enumerable type for another, and thus lots of opportunities to recoup that complexity. The first time the interface lets me write one function instead of two (say, a batch processing function that can deal with objects in an Array or those being streamed from an IO, because they both implement foreach), that's a win.

I have to think a lot harder to come up with a situation where I have an object that might be an Option and might be a Future, and I don't care which type I have. It doesn't seem like it would ever come up in business logic, so we're probably looking at some sort of mid-level abstraction that depends a lot on the code on both sides of it. Maybe that's why writing a decent, concise tutorial is so hard.

I like your article. You make explicit that being able to use Options and Futures interchangeably is the objective, and your example code actually depends on all the monad behaviors instead of just flatMap (I think - Scala is a foreign language to me). I still don't know how often those circumstances come up in practice, but maybe with experience I'll start seeing the patterns more often. Thanks again.

[lmm]

> However, there are lots of situations where one may want to substitute one enumerable type for another, and thus lots of opportunities to recoup that complexity. The first time the interface lets me write one function instead of two (say, a batch processing function that can deal with objects in an Array or those being streamed from an IO, because they both implement foreach), that's a win.

> I have to think a lot harder to come up with a situation where I have an object that might be an Option and might be a Future, and I don't care which type I have. It doesn't seem like it would ever come up in business logic, so we're probably looking at some sort of mid-level abstraction that depends a lot on the code on both sides of it. Maybe that's why writing a decent, concise tutorial is so hard.

I see where you're coming from. Monads are parametricity rather than substitutability: an Option and a Future are similar to each other in the same sense that a List<Int> and a List<String> are similar to each other. You'd never have a situation where you didn't care if you had a List<Int> or a List<String>, but there are still functions like sort() or find() or foreach() that are useful at the generic level.

And it's hard to do a good concise monad tutorial because they're an abstraction over an abstraction. Really a good tutorial would probably need three different examples - say Option, Future, and Writer - but then you'd probably need three examples for each of those to show why they're a worthwhile thing to be using in the first place.

> your example code actually depends on all the monad behaviors instead of just flatMap (I think - Scala is a foreign language to me).

flatMap/bind is the most important part - if you have a well-behaved (i.e. associative) flatMap you're 90% of the way to being a monad. I did use "wrap" for the base case (empty list), which is the kind of place it usually comes up. Don't get me wrong, it is important, but you can go a long way with just flatMap.

> I still don't know how often those circumstances come up in practice, but maybe with experience I'll start seeing the patterns more often.

Once you're used to them you see them everywhere - it's such a simple and general interface. I said anywhere you'd have a cross-cutting concern; another heuristic might be anywhere that you'd like to use the command pattern if you were "doing it properly", but there would be so much overhead to actually defining a command type. Simple things like authorisation - http://blog.sigfpe.com/2007/04/homeland-security-threat-leve... is slightly a joke, but it's something you can apply genuinely in an application that has some concept of being logged in at different privilege levels.

[tsimionescu]

Well, this shows one simplistic partial syntactic solution to a few common problems. Partial because it does not handle errors in the continuation example, and simplistic because it only works at the function level, it's not clear how this would look like when 'distributed' through a large code base, like cross-cutting concerns typically are. Not to mention, it's not clear how to compose all of these separate solutions - how will the do notation work if you have a list of continuations that can each return optional values that return errors if something is not authenticated?

Note, I'm not claiming that these problems are not solved by monads. I'm arguing that the article you showed gives me no information on how really complicated problems are actually solved, it just shows a neat bit of syntax sugars that works on a few toy problems (again, that's what's shown, not claiming that this is all that 'do' is).

[lmm]

> Well, this shows one simplistic partial syntactic solution to a few common problems.

It's not just syntactic - the different cases genuinely do implement a common interface.

> Partial because it does not handle errors in the continuation example

Nor does the non-monad version, so it's a fair comparison.

> simplistic because it only works at the function level, it's not clear how this would look like when 'distributed' through a large code base, like cross-cutting concerns typically are. Not to mention, it's not clear how to compose all of these separate solutions - how will the do notation work if you have a list of continuations that can each return optional values that return errors if something is not authenticated?

Sure, but again, a problem that exists even more strongly if these are implemented as (non-monad) language features (e.g. if my language has both continuations and exceptions, what happens when some continuation-based code throws an exception).

The point is that you can get rid of a whole bunch of complex language features and language keywords, and write everything in terms of plain functions and values (in particular, the result is that you can refactor fearlessly because everything follows the normal rules of the language). You don't have to look at anything large scale to see the benefit of that.

[tsimionescu]

> It's not just syntactic - the different cases genuinely do implement a common interface.

Do notation is a syntactic solution, it is not an object of the runtime.

> Nor does the non-monad version, so it's a fair comparison.

The initial version does explicitly handle errors. The async/await based version also handles errors (any exceptions thrown by intermediate functions will be thrown by whoever is trying to use the results). What will the do-notation continuation version do with any errors returned by the intermediate functions?

> Sure, but again, a problem that exists even more strongly if these are implemented as (non-monad) language features (e.g. if my language has both continuations and exceptions, what happens when some continuation-based code throws an exception).

Not really - it is usually very clear how exceptions integrate with other control flow features such as continuations or, much more easily, promises or async/await (full continuations a la scheme call/cc happen to not work with exceptions, but they break any other control flow feature anyway, so that's not very surprising).

> The point is that you can get rid of a whole bunch of complex language features and language keywords, and write everything in terms of plain functions and values (in particular, the result is that you can refactor fearlessly because everything follows the normal rules of the language). You don't have to look at anything large scale to see the benefit of that.

I completely disagree with the idea that language keywords make refactoring difficult in any way, or that a language with less syntax is always easier to work with. Even the designers of Haskell disagree with this, as they have included both do-notation and list comprehensions, instead of letting programmers simply use bind and return. Even in Lisp, which doesn't have any syntax sugar in the base language, designers always add DSLs and Reader macros to add syntax back into the language.

[lmm]

> Do notation is a syntactic solution

But it's coupled to a monad interface in the language (or equivalent concept e.g. a typeclass in Haskell). It's not just a superficial similarity of syntax, you can write reusable functions that work for any monad, and the monad abstraction is very useful even if you prefer to write https://fsharpforfunandprofit.com/rop/ - style code that doesn't make use of do notation at all. The syntax looks the same because the values its working on implement a common interface - do notation itself is a very lightweight sugar that translates directly into interface methods.

> The initial version does explicitly handle errors. The async/await based version also handles errors (any exceptions thrown by intermediate functions will be thrown by whoever is trying to use the results). What will the do-notation continuation version do with any errors returned by the intermediate functions?

Neither the async/await nor the promise-based version includes any explicit error handling. So you're applying a real double standard; how is it any more or less clear what the monadic version will do versus what those versions will do?

> Not really - it is usually very clear how exceptions integrate with other control flow features such as continuations or, much more easily, promises or async/await

Having had to deal with some exceptions in sequence-continuation and async/await-based Kotlin code only last week, I have to disagree. Often there is more than one possible way to combine two effects, but if those effects are language features then the language designer will have picked one and you have to hope it's the behaviour you wanted. (E.g. if a parallel loop throws an exception, does this fail the other cases or just that one result? If an async function throws an exception before the first await, does the exception happen immediately or get suspended? There's no right answer, there are use cases where each behaviour is what you want).

> I completely disagree with the idea that language keywords make refactoring difficult in any way, or that a language with less syntax is always easier to work with.

It's not the syntax that's the issue, it's the semantics. In my experience the majority of production regressions caused by code changes (most production bugs tend to be caused by config changes, but that's a separate rant) come down to a particular category of language feature: things that can't be understood as plain old functions or values (to use the FP jargon, things that violate referential transparency). E.g. hoisting a value out of a loop changes the behaviour because the method that computes the constant was actually throwing an exception. E.g. inlining a method means a transaction decorator isn't applied. E.g. pulling out a common expression into its own function means it gets executed on a different thread. So replacing a language feature with a function that I can read the source code of (and, even more importantly, can be confident will follow the normal rules for functions in the language) is the best way I've found to eliminate whole categories of bugs at a stroke.

> Even the designers of Haskell disagree with this, as they have included both do-notation and list comprehensions, instead of letting programmers simply use bind and return.

What I'm arguing is that dedicated language constructs have a cost - of course that cost is sometimes outweighed by the benefits. But if you can replace five different special cases with a general case that covers all of them (and isn't five times more complicated to understand), that's a huge win.