| I think any argument against the possibility of developing AGI is going to have it. The argument has to be either: 1) there’s something non-material about human intelligence (basically, there’s a soul), or
2) something about the processes that created a completely material human intelligence is impossible in principle to reproduce, either implicitly or explicitly. (1) has the obvious religious angle, but (2) tends to be what’s trotted out when (1) is too overly religious. With (2), the usual supporting reason is that the conditions are too complex. The problem is that the fundamental rules are just those of physics, which are “simple”. And we have to remember that the initial conditions of the universe were also “simple” and not intelligently set up in a way that could be predicted to create intelligence. It was just a bunch of initially formless matter evolving over time. By closing the door on even implicit use of physics (which created our own intelligence), which we don’t know enough to rule out completely, there’s the feeling that there’s some kind of magic dust that has to be part of the process or the initial conditions. That would disagree with our current understanding of the laws of physics and early development of the universe. Ultimately, the real motivation is the desire to maintain the feeling that humans are somehow “special” in the universe. |