[mhawthorne]

Agree that CQRS seems like a useful way to partitions writes from reads, aka slower requests from faster requests, to avoid many fast requests waiting behind a few slow ones in line.

But even if you shifted reads to one or more caches or read replicas, wouldn't those also have queues that will fill up when you are under-provisioned?

Note that I'm using the term "queue" pretty loosely, to include things like Redis' maxclients or tcpbacklog, or client-side queues when all connections are in use.

[zamalek]

Absolutely. That's typically a good problem to have :). Hopefully you would had gradual enough growth to implement elastic scaling before this is an issue, but you're definitely eventually screwed and have to outright copy what the likes of FAANG do - your startup is a unicorn at that point, so you'd probably already have the talent hired.