[kevsim]

This sounds totally reasonable for a company like Auth0 that has a) a big engineering team and b) tons of people who are depending on their APIs not being slow.

Do you think it's easy enough to get this sort of machinery set up even for small startups?

[toast0]

If you're collecting stats anyway, it's not usually too hard to collect a latency metric and process it.

There's room to argue about methodology (ie client vs server latency, where and how timing probes happen), but just pick something and try to measure no more than 10 phases of the requests (timing probes have costs too), and see what happens. After you have a baseline, you can set targets and/or see how releases change your data.

[hyperman1]

Why not? It's more a question of attitude than technology.

Measure it any way you want. Take an old pc, drop some monitoring package like zabbix on it, and let it poll the APIs. Log entry and exit time. It wont be perfect, but even the lowest effort measurement is better than nothing at all.

[stonemetal12]

It depends on how fancy you want to get. It could be as simple as a unit test that runs some scenarios a dozen times under a timer and fails if 90 percentile time is greater than the target time.