[longerthoughts]

So basically AI abiding by Goodhart's law (https://en.wikipedia.org/wiki/Goodhart%27s_law). I wonder how much of this goes undetected in other applications due to poor objective definition.

[olooney]

It's incredibly common. For example:

https://docs.google.com/spreadsheets/u/1/d/e/2PACX-1vRPiprOa...