[cageface]

From an interview with the father of Haskell:

http://www.infoq.com/interviews/armstrong-peyton-jones-erlan...

But it turned out to be very hard to turn that into actual wall clock speedups on processes, leaving aside all issues of robustness or that kind of stuff, because if you do that style of concurrency you get lots of very tiny fine-grained processes and you get no locality and you get very difficult scheduling problems. The overheads overwhelm the benefits, in short.

It may be true that FP does make concurrency easier, but I don't think this has been demonstrated to be generally true yet and it's certainly not as simple as just eliminating side effects from your code.

[lukev]

While true, this is somewhat disingenuous. When people speak of concurrency in functional languages, they don't mean that each function executes in its own thread (even if it can, according to the semantics of the language). As this quote points out, the overhead of that is absurd, though maybe we'll get there someday with specialized hardware.

What functional programming does mean right now is that it's much easier to chop your program up into reasonable-sized chunks: big enough that it's worth the overhead, small enough that you can distribute as widely as possible.

In other words, if you think of a functional program as a tree and then expect to run each leaf in its own thread, the overhead will vastly outweigh the benefits. But the functional semantics also allow you to run each of the dozen main branches in separate threads with little effort.

I do this in Clojure all the time.

[cageface]

Sure but this is also what you generally do in an imperative threaded app. Break the work up into discrete jobs and use a producer/consumer model to parallelize it. This has to be done carefully and usually requires an intimate understanding of the flow of the computation. It might be easier to do this in a functional language but their advantages tend to be mostly at the micro level and the hard stuff tends to be at the macro level.

I honestly think that the advantages of FP for concurrency will turn out to be relatively minor and that FP is more interesting as a means of improving code reuse and reducing bug counts. I'm not convinced that the current crop of FP languages are a win in those respects either though, taken as a whole.

[lukev]

Well, yeah. There's no silver bullet. But it's much easier to start lopping off branches from the tree if there are no unmanaged side effects.

Programming languages of the future need to enforce thread safety and mutation semantics as well as they do type safety. Functional programming is a step in that direction.

Nobody's necessarily claiming that a concurrent Clojure/Erlang/Haskell app is faster than concurrent C++/Java one, but they're definitely safer and easier to write.

[cageface]

I wonder if the kind of concurrency people have been focused on in these kinds of conversations is really going to turn out to be that important. It seems to me like a lot of the really cpu-intensive problems people are trying to solve are way beyond the scale of a single machine - the kinds of problems you need a big hadoop-style farm to tackle. How many apps are there left that really need maximum throughput in a shared memory architecture?

[Turing_Machine]

"When people speak of concurrency in functional languages, they don't mean that each function executes in its own thread"

Erlang does do that, more or less, although Erlang processes are much lighter weight than what we usually think of as a "process" or "thread".

[lukev]

As an added note, I'm not aware of anyone doing this yet, but it would be really cool to have a VM that did JIT-like analysis of purely functional code as it runs to determine optimal points for thread segregation.

[herdrick]

Really? Can you post some profiling data?

[neilc]

It may be true that FP does make concurrency easier, but I don't think this has been demonstrated to be generally true yet

Perhaps not generally true, but I think MapReduce and LINQ are two prominent examples of how a "functional programming-like" model lends itself to easy parallelization and distribution.

[microtonal]

And there are also enough 'prominent examples' for impure imperative languages. For instance, OpenMP makes parallelizing some loops in C/C++ a walk in the park, and I have parallelized some of my programs by adding only a few pragmas.

The 'FP makes concurrency easier' is a often-used selling point, but I think the jury is still out. First of all, many functional languages are impure, and do not have this advantage in the same manner that e.g. Haskell has. Second, purity can be an expensive trade-off due to the copying involved. For lots of things (e.g. fast linear algebra) you'll still want to work in an impure world.

I think a better selling point of some functional languages is clarity. For instance, some class of problems can be solved very elegantly in Haskell and ML using algebraic data types, pattern matching, etc. But then again, some other classes of problems can be solved elegantly in Prolog.

[neilc]

And there are also enough 'prominent examples' for impure imperative languages. For instance, OpenMP makes parallelizing some loops in C/C++ a walk in the park, and I have parallelized some of my programs by adding only a few pragmas.

I'm curious -- were the programs that were easily parallelized with OpenMP largely data parallel to begin with? I haven't used OpenMP extensively, but I'd suspect that the ease of parallelizing a program with OpenMP probably varies inversely with the amount of explicit coordination/synchronization/update of shared state that the program requires.

[microtonal]

Mostly, yes. This particular program evaluates effectiveness of features given an existing log-linear model. So, you can evaluate the effectiveness of features in parallel. For each feature, you can also partition the training data, process the the data in parallel, and apply a reduction step[1].

But 'map' in MapReduce is also a typical data-parallel task.

[1] In practice, there is a trade-off: the vectors are usually so large for the average training set, that you do not really want to copy them for memory-efficiency, so the mapping and reduction are interleaved, requiring some locking.