Hacker News new | ask | show | jobs
by jupp0r 974 days ago
> it doesn't sound they really sharing data with each other, it looks like your logic is well lineralizable and data localized, and you can't implement access to some global hashmap in that way for example.

Yes, because data can have thread affinity. Data doesn't need to be shared by _all _ connections, just by a few hundred/thousand. This enables connections to be scheduled to run on the same thread so that they can share data without synchronization.

> I run this(10k threads blocked by DB access) in prod and it works fine for my needs. There are lots of statements in internet about overhead, but not much benchmarks how large this overhead is.

The underlying problem is old and well researched: https://en.wikipedia.org/wiki/C10k_problem

1 comments

> Data doesn't need to be shared by _all _ connections,

data doesn't need to be shared in your specific case, not in general.

> The underlying problem is old and well researched: https://en.wikipedia.org/wiki/C10k_problem

wiki page doesn't mean it is well researched, where can I see results of overhead measurements on modern hardware?

> wiki page doesn't mean it is well researched, where can I see results of overhead measurements on modern hardware?

Here is how this works: at the bottom of the wiki page, there are referenced papers. They contain measurements in modern hardware. You read those, then perhaps go to Google and see if there is any newer research that cites those papers.

If you don't feel like reading papers, HN has a search bar at the bottom that yields a wealth of results: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...

I spent short time looking and found that most papers are very outdated or don't have relevant info (no measurements of overhead) on that page. Give specific paper and citation or we finish this discussion.
https://blog.erratasec.com/2013/02/multi-core-scaling-its-no...

Maybe you should just take a college computer architecture course along the lines of Hennessy/Patterson. This is nothing new, I learned much of this in college 15 years ago. The problem has only gotten worse since then, computers have not become more single threaded.

my reading is that graphs in that post are just fantasized by author to demonstrate his idea and not backed by any benchmarks or measurements, at least I don't see any links on code in article and no mentions what logic he actually tried to run, how many threads/connections he spawned.

> The problem has only gotten worse since then, computers have not become more single threaded.

Computers are now can handle 10k blocking connections with ease.