Hacker News new | ask | show | jobs
by fragsworth 1136 days ago
I won't name three but the one everyone's really afraid of is the theoretical fast/hard "take-off" runaway problem. It is plausible that a smart enough AI given any instruction, without guardrails, will ultimately destroy the world in its attempt to do that thing. See the wiki on Instrumental Convergence: https://en.wikipedia.org/wiki/Instrumental_convergence and in particular, read about the "paperclip maximizer". Paperclips are just an example.

It remains to be seen if this is ultimately a real problem. If it turns out that it is, then not regulating the development of these systems in advance may be a huge mistake.