[nixpulvis]

The last paragraph really got me thinking about regression.

> raise your hand if you’ve ever been in an incident review where one of the follow up tasks was, “create a dashboard that will help us find this next time”

As a disciplined software engineer, I aspire to have each and every user facing bug captured first as an automated test. This helps form trust in the software. Ideally the users themselves can choose to write the tests and submit them for me.

This is akin to metrics. I completely agree, system-KPI metrics should be relevant and short. But there's nothing stopping you from collecting an archive of previous data experiment formulas.