[varjag]

So far there is no reason to think that superintelligence (i.e. not just cheap, abundant general human-level intelligence) is possible at all. It has to be qualitatively superior.

I mean what superintelligence supposed to do? Solve the halting problem or do other plain impossible things? Chess computers can beat a human champion with sheer firepower, but they still can't do checkmate in one move.

[pixl97]

>I mean what superintelligence supposed to do?

Compared to every other life form on Earth you are super intelligence. This super intelligence thing has already happened once. Of course this spread of super intelligence has wiped out most large mammals on this planet, 10s of thousands of other species, most of the fish in the ocean, and is adding gigatons of carbon to the atmosphere.

You can't beat chess in one move because mathematics does not allow it. Unless this said ASI develops bending 4D spacetime (in which you could beat chess in one move with some new rules) then the game simply does not have a piece that moves that way. That said ASI can be just a little smarter than us, and put us to extinction like we did with the Neanderthals.

[varjag]

> Compared to every other life form on Earth you are super intelligence. This super intelligence thing has already happened once.

You extrapolate the trend from one data point. There is no indication that we have some matyoshka type hierarchy of intelligence. It could well be binary, either intelligent or not. No reason so far to think otherwise.

> You can't beat chess in one move because mathematics does not allow it.

So what superintelligence can do that "ordinary" intelligence in right quantities couldn't? Something specific beyond theoretical grasp of "just" intelligent machine?

[psyc]

> So what superintelligence can do that "ordinary" intelligence in right quantities couldn't?

Conceive of answers to this question that we can't. Do you really believe the human imagination captures all of possibility? Why?

[varjag]

The search space of all possibilities is infinite, and hence beyond the grasp of any intelligence. I firmly believe human imagination and all other forms of intelligence are restricted by mathematics and physics of our universe. And as such, the fundamental set of solvable problems remains the same.

Would be nice to hear some specific arguments against instead of Penrose-like quantum handwaving and pet analogies.

[eli_gottlieb]

>Do you really believe the human imagination captures all of possibility? Why?

Well yes, because we are already generally intelligent. The problem is not to have the right hypothesis space built into our wetware, but to locate correct (action-guidingly veridical) hypotheses within the existing hypothesis space based upon sense-data.

[benibela]

Human imagination is Turing-complete.

If the AI runs on a normal computer, it is also Turing-complete and not beyond.

So they have the same limits

[psyc]

The difference is in the usual practical constraints of time and space.