Hacker News new | ask | show | jobs
by kiitos 874 days ago
TCP connections are bottlenecked not just by the browser/client, but also at the load-balancer/server. Modulo SO_REUSEPORT, a server can maintain at most 64k active connections, which is far below any reasonable expectation for capacity of concurrent requests. You have to decouple application-level requests from physical-level connections to get any kind of reasonable performance out of a protocol. This has been pretty well understood for decades.
1 comments
otterley 874 days ago
That limitation was overcome over 20 years ago with the invention of Direct Server Return (DSR) technology, since the remote IP becomes that of the actual client. (This also helped relieve pressure on load balancers since they don't need to process return traffic.) Another way to overcome this would be to use netblocks instead of IP addresses on both the load balancers (upstream side) and the server side (so the LB has multiple IPs to connect to and the server to respond from).

The benefit of DSR became mitigated a bit after CGNAT (in the IPv4 space anyway) began to be rolled out, since it can masquerade a large group of clients behind a single IP address. (CGNAT poses other, different problems related to fraud and abuse mitigation.)

link
kiitos 874 days ago
It's not a question of IP addresses, it's about connections.
link
otterley 873 days ago
Which limit, exactly, are you referring to? Both load balancers and backend servers can juggle millions of concurrent connections nowadays. You mentioned a 64k connection limit but that’s not a hard file descriptor limit, nor does the 65536 port limit apply if the source and/or destination IPs differ.
link
kiitos 872 days ago
> Both load balancers and backend servers can juggle millions of concurrent connections nowadays.

Maybe with SO_REUSEPORT, but not in general.

A TCP connection is identified by a 5-tuple that requires a unique port for both the client and server. TCP represents ports as uint16s, which means the max number of possible ports per address is 65536.

tl;dr: 1 server IP address = no more than 64k incoming connections

link
otterley 871 days ago
Yes, I'm aware that 4-tuples must be unique. And so, by having a LB and/or server bind to more than one IP address, you can easily overcome that limit.
link
kiitos 871 days ago
If you have multiple client-accessible IP addresses available to a server, I guess? But that's in no way common?
link