A server with 100K TCP connections isn't much different, and you might be surprised (or maybe not) by how extensively dynamic data structures are used in the TCP stack implementation. I'm not sure all of them can be made 'cache-friendly,' and embedding all layers together would make the code unmaintainable. Intensive access to external memory is inevitable when accessing a large enough number of unrelated states.