It's all about the load though. SO is probably 95% Read-Only which makes sense for removing the cache layer. If you had a more writes, then they would need an external cache to offset the read load.
I don’t follow. Holding the total server load constant, why wouldn’t a read-heavy workload benefit more from caching than a more balanced read/write workload?