[toast0]

It's easy to just get 4TB of ram if that's what you need; I haven't scoped out what you can shove into a cheap off the shelf server these days, but I'd guess around 16TB before you need to get fancy servers (Edit: maybe 8TB is more realistic after looking at SuperMicro's 'Ultra' servers). I think you'd need a very specialized applicatjon for 100M connections per server to make sense, but if you've got one, that sounds like a fun challenge; my email is in my profile.

Moving 100M connections for maintenance will be a giant pain though. You would want to spend a good amount of time on a test suite so you can have confidence in the new deploys when you make them. Also, the client side of testing will probably be harder to scale than the server side... but you can do things like run 1000 test clients with 100k outgoing connections each to help with that.