Hacker News new | ask | show | jobs
by otterley 878 days ago
That limitation was overcome over 20 years ago with the invention of Direct Server Return (DSR) technology, since the remote IP becomes that of the actual client. (This also helped relieve pressure on load balancers since they don't need to process return traffic.) Another way to overcome this would be to use netblocks instead of IP addresses on both the load balancers (upstream side) and the server side (so the LB has multiple IPs to connect to and the server to respond from).

The benefit of DSR became mitigated a bit after CGNAT (in the IPv4 space anyway) began to be rolled out, since it can masquerade a large group of clients behind a single IP address. (CGNAT poses other, different problems related to fraud and abuse mitigation.)
1 comments
kiitos 878 days ago
It's not a question of IP addresses, it's about connections.
link
otterley 878 days ago
Which limit, exactly, are you referring to? Both load balancers and backend servers can juggle millions of concurrent connections nowadays. You mentioned a 64k connection limit but that’s not a hard file descriptor limit, nor does the 65536 port limit apply if the source and/or destination IPs differ.
link
kiitos 876 days ago
> Both load balancers and backend servers can juggle millions of concurrent connections nowadays.

Maybe with SO_REUSEPORT, but not in general.

A TCP connection is identified by a 5-tuple that requires a unique port for both the client and server. TCP represents ports as uint16s, which means the max number of possible ports per address is 65536.

tl;dr: 1 server IP address = no more than 64k incoming connections

link
otterley 876 days ago
Yes, I'm aware that 4-tuples must be unique. And so, by having a LB and/or server bind to more than one IP address, you can easily overcome that limit.
link
kiitos 876 days ago
If you have multiple client-accessible IP addresses available to a server, I guess? But that's in no way common?
link
otterley 875 days ago
It's quite common in Kubernetes deployments, where each server in a separate Pod binds to a separate IP address.

And, as I said before, with DSR, there's a broad diversity of client IPs, so a single server address doesn't typically cause concerns with 4-tuple exhaustion.

link