[NtochkaNzvanova]

So... are the humans being used to train AI or not? The article seemed vague on this. It makes sense that they would be, if only to eventually lay off the humans to save costs. (Assuming that the AI inference is actually cheaper than employing humans to do it. Maybe if the models are complex enough it wouldn't be.)

The thing I find confusing is this: if an AI were being trained, I would expect it to quickly get good at distinguishing the extremely obvious bad content from obvious acceptable content. So over time, you would expect the human moderators to only need to make calls about borderline cases. I.e., over time what the humans see will tend to become less horrible, and the AI will eventually get good at making those calls as well. If this isn't happening, why not?

[Terr_]

I suspect part of the problem is that AI uncertainty and failure still doesn't always work the way humans expect, so what's "borderline" for the AI isn't necessarily borderline for human, and vice-versa. This leads to a problem when it comes to reliably detecting and escalating issues. (Especially when something slipping through might have large legal repercussions.)

Imagine a recognition task where both machines and humans have the same X% false-[positive/negative] rate, but where there's zero overlap between the cases that each system considers uncertain. Failures by the algorithm would proceed without being flagged for review, and all the human reviews would be of "obvious" things.