Hacker News new | ask | show | jobs
by jonathanstrange 569 days ago
No professional Go programmer would spawn 1M goroutines unless they're sure they have the memory for it (and even then, only if benchmarks indicate it, which is unlikely). Goroutines have a static stack overhead between 2KiB to 8KiB depending on the platform. You'd use a work stealing approach with a reasonable number of goroutines instead. How many are reasonable needs to be tested because it depends on how long each Goroutine spends waiting for I/O or sleeping.

But I can go further than that: No professional programmer should run 1M concurrent tasks on an ordinary CPU no matter which language because it makes no sense if the CPU has several orders of magnitudes less cores. The tasks are not going to run in parallel anyway.

1 comments

The basis for running 1 million concurrent tasks is to support 1 million active concurrent user connections. They don't need to run in parallel if async is used. As shown, Rust and C# do well. How would you support it in Go?
The servers I use have limits far below 1M active connections, realistically speaking about 60k simultaneously active connections. So I can't really answer that question. However, it's easy to find answers to that question online [1]. Go is not forcing you to spawn Goroutines when you don't really need them. As I said, the correct way in Go is to use worker pools, the size of which depends on measurable performance because it is connected to how much i/o each Goroutine performs and how long it waits on average.

[1] https://www.freecodecamp.org/news/million-websockets-and-go-...

Pay extra $1/month for bigger server? The numbers are really small here.
You obviously missed the chart. The memory usage difference isn't small; it's astronomical as far as scaling goes.
In what world is the cost of 2.5 GB of RAM per 1 million connections an issue? Are you telling me there's a service in this world handling, for example, 100 million active connections, and they can't afford 250 GB of RAM? It's not the 90's anymore.

And we're talking about a naive microbenchmark. If you were actually building a service like that in Go (millions of active connections) and you were very concerned about memory usage, you wouldn't be naive enough to use a goroutine for every connection. Instead, you would use something like gnet or another solution based directly on epoll events, combined with a worker pool.

> If you were actually building a service like that in Go (millions of active connections) and you were very concerned about memory usage

Or, much more likely, pay a few tens of dollars per month for a second server and start scaling out.

Go is a friendly language and it is well liked, but it is inappropriate when the absolute best performance saves more money than the additional developer salaries needed to write it in a higher performance language. An example is for deep learning or other big numerical work, where you'd be wasting expensive hardware resources if using Go. Perhaps the one million concurrent users will however be fine with Go.