[akvadrako]

I'm not sure what point you are trying to make. It's possible to dismiss an idea without providing an alternative. Yes, finding a reward function is equivalent to figuring out what we care about. Both are about as hard as teaching a bacteria to play piano.

The goal is avoiding unsafe AI. The reason such pointless efforts are wasted on this approach is we don't have a good alternative. The only thing I can think of is delaying it's creation indefinitely, but that's also a difficult challenge. For example, in the Dune books, the government outlaws all computers. That might work for a while.

[argonaut]

Let me elaborate. It is easy easy easy to nitpick and find holes in someone's proposals, someone's problem statements, and someone's goals in life.

Statements are adding noise and less than nothing of value if they just consist of telling people they are working on the wrong thing... and not proceeding to tell them what they should instead be working on, and giving clear positive reasons why (instead of negative reasons someone should not be working on something).

Incidentally this is a broader problem with HN discourse.

[stevetrewick]

From my cold, dead hands.