[arisAlexis]

The denial of x-risk is crazy here. This is literally a demo of what researchers like Hinton and Bengio are afraid of but most comments don't believe it happened and the other think that it's not a big deal. The human psyche never ceases to amaze.

[utucuro]

To be frank, I'm not afraid of AI, I'm afraid of the exact opposite - natural stupidity. This story clearly shows that too - "algorithm trained to prioritize scoring points tries to do so at any cost" It's not even emergent behavior, it's badly defined goals...

[arisAlexis]

Read up about the "alignment problem". It is one that ai researchers cannot Crack and its literally what you wrote. The idea that we could never in any way define goals align with us.

[rbanffy]

We have been living with this problem of poorly defined goals for ages. Computers do what they are told to, not what we want them to. I built a career on translating what people want computers to do into what computers need to be told.

I suspect we'll hear of Dr. Susan Calvin more and more in the future.

[utucuro]

I already know about the efforts about AI alignment - this is the reason I'm afraid about natural stupidity. I really do not want to see the alignment issue solved in my lifetime.

[arisAlexis]

Cringe comment: the reason why will not see it both you and me will be the same if it doesn't get solved :)

[tedunangst]

It didn't happen, so believing it didn't happen shouldn't be that crazy.

[jazzyjackson]

ur in a cult, complete with eschatology and in-group jargon

its not a big deal if a computer program arrives at the conclusion that its operator should be killed, its just extremely stupid to give such programs access to weapons systems.