|
|
|
|
|
|
by pmontra
1104 days ago
|
|
|
This implementation is probably a little different but we were using long poll in the late 90s and early 2000s. The problem was that you were committing one thread (or worse, one process) to each client. That obviously doesn't scale unless threads are extremely light on RAM and either the OS or the runtime support a large number of them. I remember that a way out was using continuations. Jetty was a Java application server that supported them (random link [1]) One thread -> many connections. I didn't investigate how Django is implementing this now but CPUs and RAM are still CPUs and RAM. [1] https://stackoverflow.com/questions/10587660/how-does-jetty-... |
|
|
This is mostly true today as well, although we do have beefier machines and more efficient thread pooling.
I did some personal research for infrequent real-time notifications. My conclusion was that many stateful connections are poorly memory-optimized by language runtimes and reverse proxies. Even with lightweight tech like Golang, gRPC, nginx, etc, it’s hard to push anything less than 30 kB for an idle conn, mostly from thread/goroutine stacks, user space buffers and (easy to overlook) a bunch of HTTP headers and TLS handshake state that often remain after establishing the conn. That’s without any middleware or business logic wants their share of the cake.
The only mature project I found that really took this stuff seriously is uWebSockets. It’s extremely lightweight, around an OOM better than most alternatives. Highly recommend – they also have a socket library so you can implement other protocols if needed.
Anyway, it’s important to be aware that massive amounts of long running connections is not just about adding a websocket/SSE library. Chances are you’ll need a mostly-separate serving stack for that purpose.