[gavinray]

What is "referential transparency"?

> "by hiding the extraction of the value contained in the monad and passing that value to a user supplied lambda that takes a value of the type of the value contained in the monad and returning a new monad containing a different type"

I'm not sure I follow. Is this like saying I have a function:

    let myFunc = (arg1: SomeType): NewType => {}

And it takes SomeType, and returns NewType? What does it mean to "hide the extraction of the value"?

[gmfawcett]

I wrote a Reddit post a while back that might help. (The parent comment has since been deleted -- the poster was asking whether using monads in Haskell was kind of like "dependency injection", i.e., passing in all the information that a function might need, as arguments to the function.)

https://www.reddit.com/r/programming/comments/593ud7/a_taste...

[jackcviers3]

The reply by louthy is great, but it doesn't answer your first question:

> What is referential transparency?

A function reference that can be replaced at all call sites in a program with its body with all of the references to its arguments replaced with references to the call site arguments without the observable behavior of the program changing is referentially transparent.

In practice, it means that you can safely refactor lines of code into a function reference without worrying about breaking your program.

It is trivial to ensure that a function that does no side effects, like `addTwoNumbers(a, b)` is referentially transperent, and we do it all the time, and we have very little trouble reasoning about what functions like that will behave like at runtime, so we do it as much as possible anyway. We call it things like "single responsibility principle" and KISS, and it leads to more maintainable programs.

However, it isn't as easy to do with a function like `printStringLine(aMessage)` or `divideTwoNumbers(a, b)`. Obviously, if you move your print statements and the state of a program changes, you'll get different output depending on where you moved them to. And the behavior of divideTwoNumbers when the dividend is 0 is problematic. As is making a network call --> depending on when it is called, you might get a different answer. All of those things are things the programmer has little to no control over.

The monad interface provides a way to make those things referentially transparent, often storing the transformations made in a sequence of bind calls in a data structure, then interpreting that data structure via a method called `unsafeRun`. Since the return of bind is now a data structure containing the transformations to be applied, it is possible to test that the network calls, console output, and error raising and throwing logic outputs in the correct order by substituting a "logging" implementation of the monad in tests rather than an "executing" monad in tests. Then you can test the "executing" monad by sequencing two operations that produce output and checking that that output matches your expectations, rather than checking that all of the network calls in your application actually work at test time.

Additionally, since we can usually unit test our non-side-effecting functions easily, using a monad method is a type-safe, code-level indicator in code that "trouble out of our control at runtime may happen HERE." So if you have trouble, and you are making a network call in your bind/flatMap, you know where to look: in your NetworkCallMonad implementation.

Referential transparency allows us to reason about a program's behavior by just looking at the body of the function, rather than anything else around it. It's the ultimate form of encapsulation, and has the same benefits. So, now that we have a way to define referentially transparent side effects that encapsulate problems that can only happen when the stars align incorrectly, we can reason locally and test locally around all the things in the program, which means the program is simpler to reason about.

There are some consequences, though. ANY 2 Monads won't compose. List<Option<Int>>.bind((i) => i + 1) won't compile, or work. You need a MonadListTransformer<Option, Int> that knows which order the monads are composed (Does Option contain lists, or does the list contain options?) in order to do the extraction and flattening in the correct order. Of course, this adds runtime execution overhead, and in the case of two nested monads is not too difficult to acheive.

But often, when you encode a program's effects into monads, you end up with monstrosities: <List<NetworkCall<Option<Result<JsonObject>>>>>. That can get to be a mouthful, and our simple little bind definition is now several layers deep. All that means is that you need to add some methods to your Monad interface, and give them new names, so that you have a type that is a Monad and a IO and a Monoid and a ApplicativeError. Those may sound like gobbledygook names, but they are the names chosen by mathemeticians and the FP community, so to Google them you have to use the names.

Monads, and `TypeClasses` are not the only way to achieve RT programs, but they are pretty widespread, well-defined, well-tested, and well-documented interfaces that will work. In languages without higher-kinded-generics (Generics that can hold other generics), you can simulate them in any language with generics using `Box<Repr>` and `Unbox<Repr>` -- typescript does this in its fp lib, for example -- so that you can still get the code reuse out of defining simple instances that can be derived from things higher in the dependency tree.

Anyway, they have value for simplifying programs, but all of their value is derived by maintaining referential transparency throughout the entire codebase as much as possible, and by staying within a context unless there is a safe way to exit the monadic context (This is called a `Comonad` -- at allows you to get a value out of a Monad without risking behavioral changes in your application, and not all `Monads` have a corresponding `Comonad`) until the very last line of your main file. Organizing many custom monadic effects is a common concern, and a really good solution for that is to use a single monadic type that can handle all of your effect needs -- like the IO monad in haskell -- and using the good `ol interpreter pattern to implement methods that delegate to the one base monad to do their effectful work. This is called tagless final style.

One consequence is that you do a lot of wrapping and unwrapping in your code with monads. It gets tedious to call the same constructor over and over again. Haskell, and other languages with extension methods, make this easier by automatically lifting call sites into the currently used Monad context type implicitly. Otherwise, you have to call new Monad(new Monad(new MyNetworkCall("myApiAddress")).map((networkCaller) => networkCaller.callGet(1))).flatMap((result) => new MyNetworkCall("myOtherApiAddress").map((netWorkCaller) => networkCaller.post(result.user)))

a lot, which is safe, but tedious to write and read. If you have to do this because of language limitations, refactor and extract as much as possible so that your code is readable AND safe.

Obviously, if your language or a library in your language doesn't define the standard typeclasses, of which Monad is only one interface, defining them and using them can be a real pain. Using them is kind of like using any framework --> trivial programs that are small should not use monads unless screwing up at runtime is VERY expensive. A lot of business applications are neither small nor trivial, and screwing up can have dire consequences, so a lot of programs can benefit from this level of encapsulation. It's just another tool, a proven (in the mathematical sense) tool, in your toolbox that can help code quality. Referential transparency isn't a magic silver bullet or a panacea. You still have to think.

> I'm not sure I follow. Is this like saying I have a function: > let myFunc = (arg1: SomeType): NewType => {}

Not quite.

    let myFunc = <A,B>(arg1: SomeType<A>): SomeType<B> => {}

The `SomeType` wrapping the `A` is returned as a new instance of `SomeType` that now contains a `B` and not an `A`. The argument `arg1` is not mutated. The `A` it holds is extracted, transformed through user code into a `B`, and then placed back within a new instance of `SomeType`.

This obviously happens a lot in any language with generics. Like, all the time. And, every generic will implement the way it does the extraction differently, but the transformation always gets applied to the extracted value(s) in the same way, by calling the user function on the extracted value(s), then ensuring that if multiple instances are created (because multiple values were extracted) that only one instance of the generic that contains all the transformed values is returned.

Here's a simple example of it in use:

    let getCharsFromStrings(strings: List<String>): List<Char> = bind(strings)((s: String) => s.map((c:Char) => c)))
    getCharsFromStrings(List("one","two")) // outputs List<Char>('o','n','e','t','w','o')

Since each string is broken into a list of its characters, you might have expected the return to be a list of two lists of characters. A Monad does the flattening for you. In fact, in some languages it is called `flatMap`, because it applies you function to each element that is extracted and flattens the nested structure by one level, turning your list of lists of chars into a list of chars via concatenation.

[gavinray]

Monad is really just "flatMap"?

Is this basically it then:

    interface Monad  {
      flatMap: <A,B>(arg: Some<A>) => Maybe<B>
    }


    let userAges: Monad.flatMap = flatMap<User, number>(users, user => user.age)

[lmm]

That signature looks a little confused - you're saying "Monad" but you're talking about the specific case of Some and Maybe. The interface looks something like:

    interface Monad<F<?>> {
      fun flatMap<A, B>(argument: F<A>, function: A -> F<B>): F<B>
      // also need pure/point here but ignore that for now
    }

and then a specific implementation for e.g. Option looks like

    singleton OptionMonad implements Monad<Option>
      override fun flatMap<A, B>(argument: Option<A>, function: A -> Option<B>): Option<B> =
        when(argument) {
          is Some -> function(argument.value)
          is None -> None
        }

whereas you have implementations for other types in terms of those specific types too:

    singleton FutureMonad implements Monad<Future> {
      override fun flatMap<A, B>(argument: Future<A>, function: A -> Future<B>): Future<B> = ...
    }

The tricky part is that the type parameter F is itself a parameterised type, so it's a slightly higher level of abstraction than most interfaces (and one that most languages don't support).

[jackcviers3]

Yep. You got it. Monads are compared with value equality, not reference equality below. The flatMap behavior has to obey some laws:

The Associative Law states that flatMapping two functions in sequential order is the same as flatMapping over a composition of those two functions:

Monad(fa).flatMap(f).flatMap(g) valueEquals Monad(fa).flatMap(g(f))

It also has a method called return/point/pure that comes from the inherited Applicative interface:

   interface Applicative<F<?>> {
     pure: <A>(arg: A): F<A>
   }

That return/pure/point followed by flatMap has to obey the Left Identity law in a monad for it to be a valid monad:

  Monad(fa).pure(x).flatMap(f) valueEquals f(x)

The Right Identity law states that if we have a Monadic value and flatMap over the point/pure/return method we get a new monad that has value equal to the original monad:

m = Monad(fa) Monad(fa).flatMap( (v) => Monad(fa).pure(v)) valueEquals m

Those are your unit tests that you have to perform on your monad instances. If they pass, your implementation is correct, and you can rely upon it.

These laws are so that the monad interface can be composed/extended correctly with other referentially transparent interfaces, like Functor -provides map, the map must be associative, like flatMap); Apply extends Functor - provides ap, or applyParallel that allows parallel mapping over collections that can be safely parallelized. ap has to be consistent with flatMap when an implementation has a monad instance -- that is if you map over a list in parallel, the list you get out should be the same as if you flatMapped over that same list using the same function, but wrapping List() around the result, which is sequential; Applicative extends Apply - provides pure/point/return, which just constructs instances that are Applicatives from a given value; Semigroup - provides combine/concat/++, allowing two items of the same type to be combined into a new value of that type; There may be many instances for a type of semigroup (Boolean has && and || for example); Monoid extends Semigroup - provides empty, which is a value for a type when combined with another value for that type results in the other value (0 in integer addition, for example, an empty list in list concatenation, for another); traverse - provides sequence, which flips the types of wrapped values (Maybe<List<T>> becomes List<Maybe<T>> when sequence is called), traverse<G<?>,F<?>, A, B>(ga: G<A>)(f: A => F<B>): F<G<B>> -- used to define sequence, foldRight, foldMap, foldLeft, etc; And ApplicativeError - provides raiseError<F<?>, A,ERRORTYPE>(t: ERRORTYPE): F<A> and handleError<F<?>, A>(fa: F<A>)(f: ERRORTYPE => A) and catchNonFatal<F<?>, ERRORTYPE, A>(f: () => A):F<A> -- obviously these only work if your type has two members, one for success, and one for failure.

[louthy]

> I'm not sure I follow

Imagine you have a context (the monad), which contains a value of type `a`...

> "by hiding the extraction of the value contained in the monad...

Means, you don't need to care about how we get the value out of the context, it will just be done for you. That's part of the monad's job. Each different type of monad may do it differently (a List will do it many times, an Option will do it zero or one times, Async promises to give you a value, etc. ).

> "... and passing that value to a user supplied lambda that takes a value of the type of the value contained in the monad ..."

So, we have a function called `bind`, which is part of the interface for monads ... it takes the original context (monad) as well as a lambda as arguments.

Below is the signature: `m a` which is the original monad value, (a → m b) is the lambda, and `m b` is the return value, which is a monad with its contextual value type changed from `a` to `b`

    bind :: m a → (a → m b) → m b

Where you see `m a` and `m b`, they're generic types which are parameterisable on both the inner and outer type, the inner type `a` is fairly usual, but the outer-type `m` is a higher-kind and can also be parameterised, `m a` could become `List a` (which is like List<A> in C# or Java).

So, if you squint, you might be able to see that the `a` comes from within the `m a` context, gets passed to the lambda, which needs an `a` argument and returns an `m b`, then that `m b` is returned as the final result.

Each type of monad has its own implementation of `bind`. So, for a List the bind operation looks like this:

    bind :: List a → (a → List b) → List b

And so you should see that for Lists each `a` will be extracted from the first argument (so we'll have zero or many `a` values), each `a` is passed to the lambda which will return a `List b` (and so we have zero or many `List b` values), and those lists of `b` will be concatenated to give a flattened result of `List b`. And so, that's the behaviour if the List monad. Each monad will have its own rules, but must follow the signature of `bind`.

> "...and returning a new monad containing a different type..."

This just means the `a` turns to a `b` (but the `m` stays the same). Essentially it means you can do some kind of mapping of the contextual value(s) to return a new type. So, if you were to map Ints to Strings the bind signature would look like so:

    bind :: List Int → (Int → List String) → List String

In practice you can see how the lambdas returning `m b` allow for chaining of computations. The code below visits the items in two lists (listA and listB) and sums them. The lambdas close over each other to create a context for the `return` at the end of the process. Which is very close to how scoping works in imperative languages.

    bind listA (\a → 
        bind listB (\b → 
            return (a + b)))

The key to any monad is its bind function, that's where its behaviour is. But fundamentally, it's a contract, or interface which forces ordering of computation... the values returned by the lambda (which are monads themselves), clearly can't be evaluated until you evaluate the first argument to get the `a`.

And so, `m a` must run before `m b` ... combine that with `do` notation and you'll get something that looks and feels a lot like imperative code, but all of the side-effects are encapsulated and declared, and all of the logic is pure (referentially transparent).

The example above with `do` notation is:

    a ← listA
    b ← listB
    return (a + b)

Which is essentially equivalent to a nested for loop without the ceremony.

[johnday]

This is a good explanation of how the Monad typeclass maps onto data functors, but I'm not sure that it translates well onto Control functors such as (-> r).

[louthy]

I was mostly just trying to clarify and unpack the GP's quite terse statement. Definitely not trying to write a monad tutorial ;)

[johnday]

Oh, sure! In that case, mission accomplished with flying colours :)