[smcgregor]

After many protracted discussions among editors, we decided not to threshold the severity of the harm. An allegation of harm is enough. Not all incidents are created equal when it comes to severity and the taxonomy feature helps sort through these questions on top of the permalink the incident profiles provide.

[kodah]

With your other reply in context, which I think makes a bit more sense, I think you do need to think deeply about establishing a bar. I work in reliability and I've worked at companies before that didn't define "incident worthy". The outcomes are truly tragic; instead of having meaningful discussions everyone will call anything an incident that gets in their way. This makes whatever metrics you produce useless and lowers faith in your body of work.

"Harm" is an amalgamous word that not even society knows how to deal with. For instance, we apply harm reduction to some of the most chemically addictive and deadly substances on the planet, yet many of us cannot imagine applying that to things like guns.

This is to say, you've so far created a database of immeasurable things by design, which just makes it a ledger of things you care about. There are meaningful things to measure here though. You could measure outcomes, analogous to E2E tests. Count the number of humans injured, financially penalized, legally penalized, or emotionally damaged. Once you've gathered enough information you can begin to weight them by category and apply a logarithmic scale. This makes results more clear and doesn't bias them in terms of people's perceptions or preferences.

[RowanH]

You realise this is fearmongering on AI right? It's literally trying to find anything that AI might be connected with as "bad" to paint AI in a negative light. I'm sure you can/are hiding behind "well readers can draw their own conclusions".

But in this day and age you just need 4 seconds to look at a page of headlines, numbers, and a conclusion will be drawn.

I'd hardly conclude that email as an "incident". There's incidents every day of people using technology and screwing up - we don't have a "Reply-AllIncidents.com"

I'm all for highlighting and examing real, scary, tangible problems with AI. But it feels like you've got the bar way too low here.

[smcgregor]

The line is not so clear as you are indicating. If I produce a system that makes 7 billion people feel slightly more depressed, then that system will in all probability contributed to a non-zero number of suicides. The unfortunate matter is that with fantastical scale there comes a need for systems that can understand the long tail of outcomes because there are real lives and impacts in the tails.