[Robin_Message]

If you think shared state is hard to scale and you are arguing for message passing, then what's wrong with Erlang? In fact, you can't conflate Erlang, Haskell, O'Caml and Scala together as FP.

By being immutable by default, FP makes message passing simpler and in some cases forces you not to do shared state, so FP helps you do concurrency that way. Closures are old news and are in almost everything now anyway (I'd include Java anonymous inner classes, they just have a nasty syntax, although good for grouping methods, e.g. mouse events).

Other than immutability by default and closures, what makes a language functional anyway? Because if you take the languages you suggested together, that's all I see them having in common.

[yummyfajitas]

...what's wrong with Erlang?

Strings:

    1> [98, 114, 111, 107, 101, 110] == "broken".
    true

Records are not real records, just compile time labels placed on top of tuples. The syntax, seriously, three different line terminators?

Python and Haskell both make great go-to languages, and you can solve a wide variety of problems in them. Erlang is the language you suffer with when you truly need massive concurrency and distribution.

[cageface]

Haskell seems to have too many rough edges to make it nice for real-world work. The inconsistent error handling (errors vs maybe vs either), weak record syntax, and ruthlessly strict stance on mutability don't sound like things I want to wrestle with on a large scale.

Scala, on the other hand, seems to strike more reasonable compromises on these points and has all the java stuff to draw on when you need it.

[kwantam]

> inconsistent error handling

I don't understand how offering several choices is synonymous with inconsistent. If you really really want "consistency", establish a policy for your code and stick with it. All three have distinct uses, and each is better for certain things than for others.

Anyhow, there's nothing particularly special about the Maybe or Either monads; you should be able to easily implement either of those yourself, and calling them a feature of the language---beyond the fact that they happen to be in the standard library---is somewhat specious.

[cageface]

Error handling is one thing you want to have handled consistently across all code, including third party libraries. I may be able to enforce a convention in my own code but the typical app uses a dozen or more libraries.

http://stackoverflow.com/questions/3077866/large-scale-desig...

[Robin_Message]

Consistent error handling is needed if you are returning 0 for success and -1 for failure. Or was it 1 for success and 0 for failure? But there's nothing wrong with a type signature that precisely describes what you can get back. Only argument I'd have with Either is it's not obvious which is the error (although left is the convention and baked into the monad), although in practice you'll be returning "Either ParseError ParseTree" which makes it rather obvious. Also, do-notation over the Maybe monad is just perfect for simple error handling, but sometimes you need to return more than None. Happily, you can switch to Either without changing a great deal. But you can't mix them easily, and that is a suckful thing about monads indeed.

[cageface]

So what do I do if I want to write a function that takes another function as a parameter, and some of the possible functions use Maybe and some use Either? What would this code look like?

[Robin_Message]

Okay, when I first saw that I thought it looked bad, like PHP or something. But then I realised strings are just [char], and that's not so bad. Compiling away the labels sounds like a good optimisation, especially given the wire format and evolvability. Three line terminators? Ugh, but no worse than "public static void".

If Erlang is suffering, why has no-one written a better front end compiler with saner syntax or a static type system?

[kenjackson]

The ironic thing is that in the HPC space, where they've done large scale message passing for decades -- the holy grail has always been large scale shared memory!

Now admittedly the programming models in HPC were ugly (MPI), but nevertheless the lack of shared state and the use of message passing certainly didn't make it easy to write high performance parallel apps.

I think the problem is fundamentally hard. And when a problem is fundamentally hard the solution to it often is "that other thing we haven't really tried yet". Until you've really tried it.

[nostrademons]

A lot could just be that the requirements are very different between HPC and typical message-passing business apps. In HPC, you want to squeeze every ounce of performance out of the cluster, by definition. If you could get rid of the message-passing overhead, that would be a huge speedup.

For most business processes, you don't care about performance all that much, you just want things to be easy to change. Message-passing works fairly well for that: it's easy to understand, composable, and let's you swap out one component for another as long as the interfaces are compatible.

BTW, message-passing isn't exactly untried. It's the basis for the service-oriented architectures that underlie Google, Amazon, FaceBook, and many other large businesses. It works very well for that problem domain.

[kenjackson]

The message passing issue with HPC codes wasn't perf overhead of the messages. Rather it was that developing message passing applications was very complicated. But message passing apps, when written correctly gave very good performance.

With that said, you're right. If you don't care about performance, or if you have very large grains of computation then message passing is relatively easy (although so is just about any model with those requirements). The question is what about when you actually do care about performance and your grains aren't so large that doing communication half-way across the planet isn't acceptable? When I'm trying to get 60FPS in my physics engine, I probably don't want to use a web service interface.

[gruseom]

That's very interesting. Can you elaborate? I assume your point is that the HPC people were forced into message-passing by the lack of large-scale shared memory, but would have preferred the latter because they found it easier to program for? And that this somewhat contradicts our fashionable ideas about how to write concurrent programs?

Also, is their situation analogous to multicore today? i.e. does many GB of RAM shared by many cores count as "large-scale shared memory" (or does it not, e.g. because of cache effects)?

[kenjackson]

Pretty much what you describe is my point. That is why the common paradigm in HPC was that you used OpenMP on a node (shared memory model) and MPI across nodes. It's not difficult to do MPI on a node, but nobody wanted to do that. And there were several attempts to put OpenMP on clusters. The most recent one I know of being Intel (http://cache-www.intel.com/cd/00/00/28/58/285865_285865.pdf).

I think it is still a little analogous. The main takeaway is that message passing doesn't make things easy. I've spent many of days debugging message passing applications. You often trade-in one type of problem for another. If anyone is interested in more detail, I can go into an example or two.

[gruseom]

Yes, please go into an example or two.

[kenjackson]

So here's a typical problem that I'd have in an HPC application. I'd have some space, represented by some 3D structure (maybe an array, or even an object-based particle system). I need to do some computation over this space -- often using some type of stencil, so in order to compute the value at <x,y,z> I need values of coordinates some distance from x,y,z.

The part that often ends up being tricky is the fact that I need to send data from processor A to processor B. And I want to send as little data as possible. So one of the first sources of bugs is that when I do my gather-scatter I make a mistake mapping a value to a coordinate. In shared-memory you never have to do this mapping back and forth, so its not an issue.

Next issue is related to the fact that I don't want to ever block waiting for data. There are a variety of models for handling this. I can do a non-blocking receive, and do some work waiting for the data to arrive. This is often another source bugs as people will often do work that depends on the new data, but they chug along without it. Add the new data when they get it, and alas their computation is already hosed.

And the last common error in this case is handing the data off to the wrong object (or processor) or being confused as to which data you're receiving at any given point in time.

Now all of these can be handled by simply being careful, and using some good programming practices. But they are just simple, if not grossly naive, examples of issues you have with traditional message passing that don't exist in shared memory.

[gruseom]

Interesting; thanks.

What you're describing sounds to me like the complexity of ferrying data around and scheduling computations is being offloaded to the app programmer. Presumably the intent behind things like OpenMP on clusters is to take care of all that behind the scenes and let the user pretend that it's all shared and program accordingly. Is that correct? If so, how far would you say such distributed infrastructure has gotten to date? Is it usable for real work, or do people end up having to learn so many limitations and workarounds that they're no better off than programming against the lower-level model in the first place?

Another question: even when there is shared memory you still have to coordinate the various processes that are operating concurrently on it so they don't clobber each other, and that, as everyone knows, is complicated too. So there is a tradeoff here. It sounds like your point is that given a choice, the HPC community would rather program against shared memory using traditional concurrency mechanisms (threads, locks, etc.) than deal with the complexities of the alternatives. Am I reading you correctly? If so, that's a pretty major point which suggests that the general-purpose programming community may be gearing up for a wild goose chase.