[darkwater]

In a world of unicorns and rainbows, absolutely. In the real world, it's as you probably already know: it's not that easy in a complex enough system.

Quick counter-example for GP: what if the 500 spike is due to a spike in malformed requests from a single (maybe malicious) user?

[laeri]

A malformed request should not lead to a 500, they should be handled and validated.

[darkwater]

Well, in the real world it might. It should trigger a bug creation and a fix to the code, but not an incident. Now all of a sudden to decide this you need more complex and/or specific queries in your monitoring system (or a good ML-based alert system), so complexity is already going up.

[laeri]

Query input validation is nearly a solved problem. If you don't I would argue this is an incident if in this case 500's are returned.

[jabradoodle]

You need to validate your inputs and return 4xx

[darkwater]

Yeah and you also shall not write bugs in your code. Real world has bugs, even trivial ones.

[jabradoodle]

If your service is returning 5xx, that is the the definition of a server error, of course that is degraded service. Instead we have pointless dashboards that are green an hour after everything is broken.

Returning 4xx on a client error isn't hard and is usually handled largely by your framework of choice.

Your argument is a strawman

[zbentley]

> Returning 4xx on a client error isn't hard and is usually handled largely by your framework of choice.

> Your argument is a strawman

That's....super not true. Malformed requests with gibberish (or, more likely, hacker/pentest- generated) headers will cause e.g. Django to return 5xx easily.

That's just the example I'm familiar with, but cursory searching indicates reports of similar failures emitted by core framework or standard middleware code for Rails, Next.js, and Spring.

[jon_adler]

True, however it also doesn’t impact other users and doesn’t justify reporting an incident on the status page.