Which limit, exactly, are you referring to? Both load balancers and backend servers can juggle millions of concurrent connections nowadays. You mentioned a 64k connection limit but that’s not a hard file descriptor limit, nor does the 65536 port limit apply if the source and/or destination IPs differ.
> Both load balancers and backend servers can juggle millions of concurrent connections nowadays.
Maybe with SO_REUSEPORT, but not in general.
A TCP connection is identified by a 5-tuple that requires a unique port for both the client and server. TCP represents ports as uint16s, which means the max number of possible ports per address is 65536.
tl;dr: 1 server IP address = no more than 64k incoming connections
Yes, I'm aware that 4-tuples must be unique. And so, by having a LB and/or server bind to more than one IP address, you can easily overcome that limit.
It's quite common in Kubernetes deployments, where each server in a separate Pod binds to a separate IP address.
And, as I said before, with DSR, there's a broad diversity of client IPs, so a single server address doesn't typically cause concerns with 4-tuple exhaustion.