[adamtj]

The server doesn't need multiple IPs to handle > 65535 connections. All the server connections to a given IP are to the same port. For a given client, the unique key for an http connection is (client-ip, PORT, server-ip, 80). The only number that can vary is PORT, and that's a value on the client. So, the client is limited to 65535 connections to the server. But, a second client could also have another 65K connections to the same server-ip:port.

edit: You may be limited by number of open sockets or file handles. It's likely a per-process limit. Google or some linux guru could help you track down what limit it actually is, but it's not the number of server ports available. It might be a number you could raise.

[jacquesm]

Right, that makes perfect sense. But it really makes me wonder why I run in to that hard limit, I've tried just about everything to get around it and no matter what I do that seems to be the magic number.

I should go and do some testing to see what's causing this, you make me feel like the solution is right around the corner.

re. your edit, ulimit will happily raise the number > 64K, all the /proc/* settings seem to be ok so that's not it, it has to be some other layer in the stack that causes this. I'll definitely spend some time on this, it's been bugging me for a long time.

edit2: there seems to be a max_user_watches upper limit to what epoll will handle.