[kasey_junk]

For sure, thats the definition of a real time system after all. But having conversations about what the long tails do to the "normal" path and what the costs (both in money and performance in the "normal" path) is quite simply something that you can't back into.

[ansgri]

Maybe I didn't understand you correctly, but you can at least have a "return error on timeout" and process that with a predictable logic. Or maybe you do have an architecture when any individual tardy request absolutely cannot impact others. After all, I come from stream processing systems where there's only few "users" with constant streams of requests, and these users are interdependent (think control modules in a self-driving car).

[kasey_junk]

What I'm suggesting is the decision on what you do in the case of long tail performance problems, is not something you can back into.

If you are going to have timeouts with logic, that has down stream implications. If you are going to have truly independent event loops, that is a fundamental architectural decisions.

None of those things match the "make it work, then make it fast". You literally have to design that into the system from jump street as it is part of the definition of "works".