What if AGI just turns out to be exactly like the current human mind is now, except more accurate at digital calculation? What if we created AGI and then it was just lazy and wanted to read about action heroes all day?
> What if AGI just turns out to be exactly like the current human mind is now
This is quite literally the point at which things start to get scary, and the outcome is highly dependent on who controls the technology.
There's the concept of a "collective superintelligence", where a large number of mediocre general intelligences working towards the same common goal jointly achieve vastly superhuman capability. We don't have to look to sci-fi to imagine what collective superintelligences are. Large corporations and governments today are already an example of this.
The achievement of artificial collective superintelligences will occur almost immediately after the development of AGI, as it's mostly a "copy and run" problem.
This is quite literally the point at which things start to get scary, and the outcome is highly dependent on who controls the technology.
There's the concept of a "collective superintelligence", where a large number of mediocre general intelligences working towards the same common goal jointly achieve vastly superhuman capability. We don't have to look to sci-fi to imagine what collective superintelligences are. Large corporations and governments today are already an example of this.
The achievement of artificial collective superintelligences will occur almost immediately after the development of AGI, as it's mostly a "copy and run" problem.