Hacker News new | ask | show | jobs
by lonestar 5813 days ago
In what way is pushing 20K requests/second "concurrency fail"?

Node.js doesn't make it easy to share state between processes, but once you're scaling to multiple processes, the jump to multiple machines probably isn't far behind. You'll need to design a distributed algorithm, or just centralize your shared state in something like Redis anyway.
4 comments
fauigerzigerk 5813 days ago
That's a good argument for a plain web app fronting a DB server. But what if you want to implement a server that keeps a lot of data in memory? What if you're not the user of Cassandra, Redis, Lucene or a specialized analytics engine but its creator?

For request/response type of scenarios (unlike batch processing) you basically have two options: (a) Use a multi-process shared memory design (or even just the file cache), which is very cumbersome to implement because you cannot use pointers and hence none of the data structures in your favorite language's library. (b) Use shared in-process state with powerful concurrency primitives. That's what clojure helps you do.

You cannot avoid to choose between these two architectures by using distribution across machines. Distribution is built on top of whatever you do on a single machine.

link
brehaut 5813 days ago
Smarter people than me have commented on this before: http://nighthacks.com/roller/jag/resource/Fallacies.html
link
dedward 5813 days ago
"The processes end up with incorrect states pretty much immediately."

Concurrency failure.

link
pjscott 5813 days ago
If you consider the multi-server case, this is a consensus problem. To solve it, you need to either use an algorithm like Paxos, or use some central synchronization point like a Redis or memcached server. Yes, node.js fails compared to Clojure when it comes to taking advantage of multiple cores on this example. Clojure's concurrency primitives are really slick. But that's not the whole story.
link
jules 5813 days ago
It would be interesting to see Clojure's transaction support extended across multiple machines.
link
alec 5813 days ago
The guy just wanted to count how many requests his server gets; seems like that ought to be easy.
link