[staunch]

Your software matters a heck of a lot more than hardware.

You pretty much have to be using a non-blocking event-based server to achieve this level of concurrency in an efficient way. Forking/threading processes will never perform anywhere near as well.

Some non-blocking event-based frameworks: Node.js/Twisted/Tornado/AnyEvent/Libevent

You probably want to create a program that all clients connect to and acts as a coordinator for the whole system. Any actual heavy lifting can be done on other servers (CPU/IO intensive tasks, etc). Each web client can establish one connection to your "coordinator" server process and additionally make whatever HTTP requests are necessary to save/fetch data, etc.

This was historically known as "The C10K problem" as in concurrent 10,000 connections. Modern hardware and epoll/kqueue make 10K pretty easy in many cases.

http://www.kegel.com/c10k.html