[rektide]

One could adopt some kind of event-based system in scala or clojure, but Erlang and Go are alike in being the lone runtimes where running millions and millions of very small messaging processes is AOK no problem for the runtime, and not something one has to work really hard for.

Fun and joy are terms applicable here because the alternative is Erlang. Zing!

Go is rather ideal for these guys use case: if it wasn't an easy win, given what a snug fit their use case is, there would've had to have been red flags abound, as this really is a nearly idealized work load for Go's use: a hell of a lot of processes which sit around doing nothing, where one occasionally gets a message and forwards it along. Perfect Go story, as Go's lightweight processes (goroutines) are ideal for this kind of Communicating Sequential Processes routing workload.

[ken_railey]

>One could adopt some kind of event-based system in scala or clojure, but Erlang and Go are alike in being the lone runtimes where running millions and millions of very small messaging processes is AOK no problem for the runtime

Is that really the case? Scala's Akka actor library seems like it would qualify just fine: http://letitcrash.com/post/20397701710/50-million-messages-p...

[rektide]

50m msg/s is super happy making times, I agree. I'd love some absurd number counting on Go's front, but Go is entirely not a word which one can get anything useful out of from Google, so fuck whomever picked that awful name & cursed their language to being unindexed in any useful way. What happened to great names like Newsqueak? :/ Anyways!

I do have a couple things to point out:

First: "JVM settings: -server -XX:+UseNUMA -XX:+UseCondCardMark -XX:-UseBiasedLocking -Xms1024M -Xmx2048M -Xss1M -XX:MaxPermSize=128m -XX:+UseParallelGC"

Second: "The test was run with 96 actors"

I am massively a fan of actors, I hound for good interesting Akka use cases being talked about. I'd also suggest digging into Kilim if one really wants to go batshit crazy in JVM world. But Akka actors are not alike what Go does- Akka went crazy fast at tossing messages around with a 2:1 oversubscription of Actors:Cores (50m msg/s was on a 48 core machine, with 96 actors), but this is a highly qualified situation- first it was done with some very well chosen black magic command line spices, and second, more importantly, it was done with a very limited number of actors, something short of the millions of tens of millions Go or Erlang might be ok with while still being stupidly fantastically high-message rate. I don't have any idea about Goroutines, but in Erlang the time sharing is mad- lightweight processes can be interrupted all over the place, allowing messages to flow in extremely robust & reliable fashions even when all the millions of processes on the system are going ape demanding CPU right now.

I have great respect for the JVM actor crew, for Akka, but read the post mentioned by & preceding the one you linked to- Akka tapped Java concurrency supreme guru Doug Lea to come in and build them an executor engine to make Akka run fast, real fast. And it does, phenomenally well, and you can reap that too, with nearly no real thinking about it cost- but if your parameters deviate too heavily, if you happen outside the bounds of this carefully set up JVM environment, who knows. What happens to your message rate when you start trying to use CPU too? What happens to the message rate when you have a million actors? Erlang, and so I've been told about Go, is, by nature, AOK with keeping millions upon millions of lightweight processes floating about passing each other messages over channels, and is not some masterfully reworked engine (the JVM) when running in those kinds of configurations.

That said, although Erlang & Go are the natively friendly to having massively concurrent systems and I still charge the JVM as not being up front designed for it, it's clearly capable. At scale it's not how fast or how good your runtime is- once you are in the runnings. For most people Akka will work great. If we were to say there's a 30% difference in msgs/s between Java and Go for X situation, if your company is make or break on that difference, your product is broke. Fix it. Learn how to scale out. Find what your development sweet spot is, what makes your workers happiest- be it Akka or Go or Kilim or Erlang or managing your own Node scale out- and dev to that. The runtime is drastially less relevant than how you get many runtimes working together to tackle the problem. And Akka here has some great answers, as does Erlang. Go, otoh, I haven't heard anything interesting out of. Node has some interesting stories, available options, but it's far from ready-to-roll out and no one has open sourced anything that threatens to gain serious traction.

[Peaker]

Haskell's runtime makes for cheaper threads than Erlang. Possibly cheaper than Go's, too.

[rektide]

Are they also in Haskell as in Erlang pre-emptable by the runtime?

There's certainly some cost in this design decision, more context that has to be swapped in and out and more state kept, but it's important capability to allow blind design & use- think Node, where everyone has to keep re-iterating how important it is to keep yielding to the event loop, to not do a lot of CPU work in a handler: irrelevant in Erlang world- in spite of the threading model not being OS threads, Erlang is happy to drop your lightweight thread on the fly when it sees fit.

[rdtsc]

> irrelevant in Erlang world

Exactly. That is a subtle distinction but for cases where responsiveness and low latency is important that is key. Another thing Erlang has is isolation of process heaps. If one process crashes, it won't affect others. No shared data structures between processes. It all goes to fault tolerance but also a major win for a completely concurrent garbage collection.

[masklinn]

> If one process crashes, it won't affect others.

Unless they're linking or monitoring one another, an other important property of tolerant and distributed system.

[rdtsc]

But that is on purpose. Sometimes you do want that. In other words they are explicitly set up to monitor/link each other.

[masklinn]

> But that is on purpose.

Indeed, but I was pointing it out because it's important to be able to do it.

[burntsushi]

> Are they also in Haskell as in Erlang pre-emptable by the runtime?

Yes. [1]

[1] - http://www.haskell.org/ghc/docs/latest/html/libraries/base/C...

[masklinn]

> Haskell's runtime makes for cheaper threads than Erlang. Possibly cheaper than Go's, too.

What's your measure for expenses? Because if it's size in memory goroutines seem more expensive than erlang processes: http://en.munknex.net/2011/12/golang-goroutines-performance.... indicates goroutines were measured at ~4k; the default start size for an erlang process is ~310 words, or ~2k on a 64b machine.

[Peaker]

1kb in ghc, iirc.