Hacker News new | ask | show | jobs
by csense 4924 days ago
Anyone who understands monads is on a class of hard drugs.

I'm still looking for someone who can present a cogent explanation of monads using only Python.

(I'm willing to replace "Python" in the above challenge with "any language I know well," which would include Python, C, C++, Java, JavaScript, and quite a few others which I won't bother to list here. Notably lacking from this list are Lisp, Scheme, Caml, Haskell, Prolog, Erlang, and most other languages whose view of the world is "weird." NB not that I'm against learning such languages; it's more a matter of available tutorials and time.)

6 comments

They're not smart to do in python. Monads are an expressive, simple pattern of coding that can be type-checked. In dynamic languages it'll just look like a lot of unnecessary line noise. In HM type systems it's a great way to help you pass contexts around in a way lets you focus on your values.

But if you just want to speak Python, let me try to translate.

---

As a motivation for monads in Python, we're going to try to make "total" Python. To do so, we have to eschew execptions. This means we'll write stuff like

    def mypop(l):
      if len(l) > 0:
        return l.pop()
      else:
        return None
which, if we pass it a list of numbers, is guaranteed to return either a number or None---I've sidestepped the empty list exception. Now let's say I want to compose this function. For instance, my list of numbers is a list of bids and I have a function where I can try buying something with my bid.

    def buy(tendered):
      effective = tendered*1.2 # there's a discount!
      if market_open():
        if effective > 60:
          return True
        else:
          return False
      else:
        return None
So I believe that buy takes numbers and returns Booleans, so I might think that

    def trybuy(l): return buy(mypop(l))
takes lists of numbers and returns a boolean as to whether I've bought it. Of course, I have to be smart enough not to send in an empty list otherwise the discount in 'buy' will cause an error. I also have to be sure that if I try buying on days the market is closed to handle the 'None'. In Haskell, we'd write these constraints into the types and values using a Maybe type.

Here's a Pythonic Maybe type (kind of not pythonic though with the magic sentinel value, but we don't want to be able to sneak 'None's in).

    class Maybe(object):

      __sentinel__ = object()

      def __init__(self, obj = __sentinel__):
        self.isNothing = False
          if obj == self.__sentinel__:
            self.isNothing = True
          else:
            self.just = obj

    def just(x): return Maybe(x)
    def nothing(x): return Maybe()
Now we can have 'mypop' and 'buy' to return Maybe types to cover the cases of empty lists and closed markets.

    def mypop(l):
      if len(l) > 0:
        return just(l.pop())
      else:
        return nothing()

    def buy(tendered):
      effective = tendered*1.2 # there's a discount!
      if market_open():
        if effective > 60:
          return just(True)
        else:
          return just(False)
      else:
        return nothing()
So now we've abstracted the 'None's out a bit. 'mypop(l)' is always a "Maybe(Bool)" and you can inspect it to determine what the case is like 'mypop(l).isNothing'. Of course, this isn't any different from using None's and checking with "x == None" except it's more explicit: if you forget that 'mypop' returns Maybes then your program will throw an error on every use... instead of just the ones where you could have gotten a None without noticing for a while. (This is where the "You Probably Already Invented Monads" idea comes from, btw.)

In Haskell we'd say that mypop has type

    mypop :: [Float] -> Maybe Float
and buy has type

    buy :: Float -> Maybe Bool
but this makes it harder to make 'trybuy :: [a] -> Maybe Bool' since buy doesn't accept Maybes. It's also pretty clear, though, that if we send in an empty list we shouldn't even attempt to buy anything (our paycheck hasn't come in yet), so let's write that as a failure mode.

    def bind(maybe_something, maybe_process):
      if maybe_something.isNothing:
        return nothing()
      else:
        return maybe_process(maybe_something.just)
and now we can write trybuy such that it respects maybes with almost no additional noise

    # trybuy :: [Float] -> Maybe Bool
    def trybuy(l): return bind(mypop(l), buy)
The pattern we've captured here---the creation of lots of explicit context to help make functions more total and fail more quickly if you misuse them and the use of 'bind' to write functions that don't need to know about context on their input---is the Monad pattern. In Haskell, we recognize that many, many kinds of context are Monadic and thus can have default compositions programmed in. 'bind' is heavily overloaded and whenever you're writing code with context you use it often to compose functions without paying the complexity cost of routing that context around.

It's obviously not a really great Python pattern. This largely has to do with the fact that Python isn't terribly explicit about what's going on: you have to remember where exceptions and edge cases can fall out. In Haskell, we just use types to encode all of that into the documentation and the type checker ensures that we never mess up.

Thanks for the lengthy reply!

So Monads are a complicated error-propagation framework that has a harder-to-understand conceptual model and results in longer code compared to Python's exceptions.

Look at any serious program in the C language -- there are enormous amounts of code devoted to error handling.

Look at the problem with Java's checked exceptions. The other day I needed to call a static method reflectively. This is what I ended up with:

  // java code to invoke a named method on a named class
  String the_class_name;
  String the_method_name;

  try
  {
     c = Class.forName(the_class_name);
     result = (Value) c.getDeclaredMethod(the_method_name).invoke(null);
  }
  catch (ClassNotFoundException e)
  {
     throw(new RuntimeException(e));
  }
  catch (IllegalAccessException e)
  {
     throw(new RuntimeException(e));
  }
  catch (IllegalArgumentException e)
  {
     throw(new RuntimeException(e));
  }
  catch (InvocationTargetException e)
  {
     throw(new RuntimeException(e));
  }
  catch (NoSuchMethodException e)
  {
     throw(new RuntimeException(e));
  }
  catch (SecurityException e)
  {
     throw(new RuntimeException(e));
  }
The point that Python's Exception model gets right, that so many other languages get wrong, is that by default, you want to pass errors to the caller.

You don't want to pass errors through the same pipeline that results flow through, since then every joint in the plumbing needs error checking. The very name "exception" is chosen to denote an "extraordinary control flow" specifically for error handling.

No, of course not. Error handling is just one example. The monad interface makes handling that problem you mention---wiring errors through the value pipeline---much easier.

And the monad part is only just the "just" and "bind" functions. Everything else is about trying to make Python more explicit with its errors: "explicit it better than implicit", right?

Monads allow for easier composition of "values in context". That context might be error, or nondeterminism, or continuation, or state, or logging, or parsing, or parallelism, or simulation, or probability, or graphs, or prolog-like logic, or streaming, or many combinations over the previous.

And in every case, you have the same interface.

If your only standard for utility is applicability to Python there's no point discussing monads or anything else that comes up in conversations about languages besides Python. Maybe the reason you haven't seen a cogent explanation of monads using only Python is the same as the reason you haven't seen a cogent explanation of how to use the clutch in an automatic car, or how to desalinate fresh water, or how to make lemonade with nothing but a jug and ice water. Maybe the problem isn't the explanation or the one producing it.
What I wish someone had explained to me earlier was that Monads are an abstraction level up from most things you ever program with. Learn specific ones, and the general picture will start to make more sense. Avoid the IO monad until you understand, in order:

+ the List monad (probably the easiest for python people, since it's very similar to list comprehensions)

+ the Maybe monad (kind of a degenerate case)

+ the State monad

And it's much easier if you do it in Haskell, trust me. "Learn You A Haskell For Great Good" is a nice easy intro to the basics.

Most information on Monads start out with IO, which I agree isn't helpful. I found one that explained it with Maybe and Either, and that was where I began to understand it.

I think Maybe is the easiest to grasp for people who work in procedural languages, since there's a very common idiom of returning a nil object if some operation doesn't succeed. You can look at Maybe and see immediately; "oh this is like a type-safe version of that". Then from there, Either makes since almost instantly, and you can sneak in List and some others before they realize they shouldn't be able to understand it :)

Since I like your order of Monadic instruction, do you perhaps have a preferred starting point for learning Arrows? I can see that it's a way to compose sequences of computation, but I don't get what problems they solve that are difficult to solve with other methods.

This is the part where I admit that I don't quite get Arrows. Anyone else?
Anyone else who doesn't get arrows? Count me in! I wish i understood this comic's joke: http://ro-che.info/ccc/12.html :(
There was a blog post that came out around that time in which it was claimed that arrows are easy to understand because they're just functions. There was a lot of backlash, and this comic is trying to point out that the reasoning behind it is backwards. Every function is an arrow, but not every arrow is a function. I believe the blog post in question is here:

http://blog.romanandreg.com/post/2755301358/on-how-haskells-...

That's about the limit of my comprehension of arrows. When you see a type (Arrow arr) => arr a b, you should read it as "an arrow from a to b", just like a function (a -> b) can be read "a function from a to b". But an arrow can encompass a monad, a function a -> m b is also an arrow, a Kleisli arrow with type Kleisli m a b. So this means you can use the arrow composition functions with normal functions as well as monadic functions, as well as arrows of other types which need not be functions at all.

Beyond that, all I know is that arrows provide composition functions that allow you to build a digraph of computations where inputs come in, get split off, passed through other arrows and recombined in interesting ways, and there is an arrow syntax which is so convoluted I've never been able to understand the relationship between it and the fundamental arrow combinators. I have never run into a situation where knowing anything about arrows was useful, other than because (&&&) is a neat combinator if you're want tuples. Practice doesn't seem to have caught up to the theory here, and things like applicative functors, which are simpler than monads in some sense, are a lot more useful than arrows, which are more complex. This blog post illustrates a case where arrows were handy, but he winds up using applicative functors in the end also:

http://jaspervdj.be/posts/2012-01-14-monads-arrows-build-sys...

It's a curious thing about functional programming that often what's simpler in theory (monoids, functors) turn out to be more useful in complex situations in the real world than what's more complex in theory.

One last note, I wrote a blog post about this in 2007 with FizzBuzz. Forgive me if it's horrible; I haven't given it much thought since, but here it is:

http://old.storytotell.org/blog/2007/04/08/haskell-arrows.ht...

Thanks for the link, I hadn't seen the second one. I felt like I almost understood what they are for, and the constructors for the arrow datatype aren't very cryptic, but then it hits the code examples, which has stuff like

    y <- readFileA "unicorns.txt" -< ()
and it's unlike anything I'm used to. I'd love to find a post that described the utility of arrows as well as his did and then maybe having a middle step where they are used without the arcane symbols, and then finally with the arcane symbols.

Your post was very clear (although your code blocks have a tiny font on my computer for some reason). The &&& and >>> combinators make sense, especially with your diagram... but in the fizzbuzz case I don't get why it's preferable to have the arrow over something like

    fizzbuzz x = combine (three x) (five x)
Although maybe it's just a case of making it simple to compose functions when there are tens of inputs or something, so the simple case wouldn't really demonstrate anything like that.
>+ the Maybe monad (kind of a degenerate case)

Hey. Hey. heyyyyy.

I've used bind + maybe monad 'ish stuff in Python to clean up code previously littered with redundant if checks.

He probably means its a degenerate case of the list monad. Which is a useful way to look at it: a Maybe value is like a list of length at most 1.

You can also look at it as a degenerate case of Either. (That is, Maybe a ~ Either () a.)

This is "degenerate" in the mathematical sense, much the same way you can think of a point as a degenerate circle.

Yes, the point is that it's just about the simplest Monad implementation that actually does something useful.

I'd observe in my code that I rarely have an enormous do block using Maybe; instead what I see a lot is use of the monadic interface to tie things together that would normally be a lot of if blocks, all on one line into one expression. You start missing that in other languages real fast. Also the Applicative instance is often quite useful, allowing you to easily tie together several things that return Maybes into a call of some sort that does the right thing if one of them is a Nothing, almost "free" from a syntax point of view.

Haha, yeah, I meant "degenerate" in the mathematical sense. Totally useful, but in explaining it you're going to get a lot of "well, it's trivial in this case because...", which isn't that helpful if you don't yet understand the underlying concepts :)
> I'm still looking for someone who can present a cogent explanation of monads using only Python.

Personally, I don't do monads in any other language than Haskell. I find monads painful even in Ocaml. I'm pretty stupid as a programmer. And --- especially when doing things such as monad transformers --- I need lots of type checking and lots of type inference to help me over my mistakes (which are pervasive).

The cool thing is that when I've written these things called "monad transformer stacks" (see Real World Haskell for what I guess is the best explanation for those), I find that when my code gets through the type checker, it really does just work. It works, even when I've lost any sense of what my code is actually doing underneath. Without the type system, I couldn't have any confidence of this, and the runtime bugs you can potentially get with monads are so subtle that I would be terrified to use them in a language like Python (note again, I'm stupid, and YMMV).

The other reason why I wouldn't do monads without types is because, to me, monads are the type. The semantics of a monad is pretty much just that type (plus some crucial axioms). Types in Haskell are much more expressive than in most languages. The way "operator overloading" works in Haskell means that by specifying types, you are often generating runtime code (think C++ templates, but not for too long). That's why you see more type annotations in Haskell compared to (say) Ocaml. Though if you want to do the same thing in Ocaml, you'll be knee deep in the even more verbose world of these things called functors.

Consider the pattern of using None (or null) as an error. You want to compute: var x x=f(x) x=g(x) x=h(x)

however, if either f,g, or h return None, then the final value of x is None. The standard way of doing this is: def f(x): if x==None: return None ...

The monadic solution is the Maybe monad. Maybe has two constructors, Just(x) and Nothing. The above code would be equivilent to var x x=Just(x) x=x.apply(f) x=x.apply(g) x=x.apply(h)

The definition of Maybe is pretty simple. The Just object takes a value of a given type, and will run that value through a given function: Class Just(): def __init__(val): this.val = val

def apply(f); return f(x)

The Nothing object is even simpler: Class Nothing(): def apply(f): return Nothing()

Using this, we can see that in our original example, once one function returns Nothing, we skip all the other functions and end up with Nothing. When a function does return something, it needs to wrap it in Just, so we have:

def f(x): ... return Just(x)

In general, to be a Monad you need to satisfy the type class Monad(): def apply(f)

Of course, using Monads like this can end up being somewhat messy, so there is much syntactic sugar around them.

>Anyone who understands monads is on a class of hard drugs.

It's so frustrating to hear you say that. Monads are such a simple concept but they're difficult to explain because they're far more general and abstract than most programming topics. The best explanation I have is that monads are a way of storing some kind of computation with a value. When the user wants access to that value, he uses a monadic operator which forces the monad's computation to be run before returning the value.

So basically monads are like a Java object that has a private field and a getter method that has some extra logic. Hey, that's really simple, you're right!

duck & runl

No, objects in Java are more complicated than monads. They include the concepts of identity, mutable state and inheritance; all of which monads lack.
Almost, it's more like an "applyer" method with some extra logic. You're never allowed to get "raw" values out, you can only pass functions inward to modify the value.

If you want to get the raw values out (and you almost always do) then you have to add some extra logic to the object above its "monadic" nature.

And yeah, it is really, really simple!

I'm trying to parse this explanation. What's the difference between a monad and overriding __getattr__ in Python?
Apples and oranges. A monad is just an idea; that of storing computations as values. This is useful because these computations can be composed together in a manner not unlike that of a pipeline in shell scripting. Different monads define different ways of composing these computations leading to vastly different semantics.

Here's a nice video that explains monads fairly simply:

http://www.youtube.com/watch?v=ZhuHCtR3xq8

Python it is. I've explained monads using Python many times.

http://www.dustingetz.com/2012/04/07/dustins-awesome-monad-t...

http://www.dustingetz.com/2012/10/02/reader-writer-state-mon...

(I'm not dustingetz though, he's named dustingetz on HN.)