[YeGoblynQueenne]

My pocket calculator is faster than a human and has better memory. I don't know that this means the same thing as "superhuman in arithmetic". I can concede that it means superhuman in speed and memory, but, arithmetic? I don't think so. What it really does is move bits around registers. We are the ones interpreting those as numbers and the results of arithmetic operations.

AlphaGo is rather different in that it actually has a representation of the game of Go and it knows how to play. I don't doubt at all that it's intelligent, in the restricted domain it operates in. But I do doubt that it's possible for an intelligence built by humans to be "superhuman" and I don't see how your one-liner addresses that.

[haliax]

Your calculator does have a representation of arithmetic too. It's those bits is moves around in registers, which are very much isomorphic to the relevant arithmetic.

Why would an intelligence built by humans not be able to be superhuman? The generally accepted definition seems to be "having better than human performance" in which case it seems we've done it many times (like with calculators).

[YeGoblynQueenne]

>> The generally accepted definition seems to be "having better than human performance"

I don't think there's a generally accepted definition and I don't agree that performance on its own is a good measure. Humans are certainly not as good at mechanical tasks as machines are -duh. But how can you call "superhuman" something that doesn't even know what it's doing, even as it's doing it faster and more accurately than us?

Take arithmetic again. We know that cat's can't do arithmetic, because they don't understand numbers, so it's safe to say humans have super-feline arithmetic ability. But then, how is a pocket calculator super-human, if it doesn't know what numbers are for, any more than a cat does? There's something missing from the definition and therefore the measurement of the task.

I don't claim to have this missing something, mind you.

>> Why would an intelligence built by humans not be able to be superhuman?

Ah. Apologies, I got carried away a bit there. I meant to discuss how I doubt we can create superhuman intelligence using machine learing specifically. My thinking goes like this: we train machine learning algorithms using examples; to train an algorithm to exhibit superhuman intelligence we'd need examples of superhuman intelligence; we can't produce such examples because our intelligence is merely human; therefore we can't train a superhuman intelligence.

I also doubt that we can create a superhuman intelligence in any other way, at least intentionally, or that we would be able to recognise one if we created it by chance, but I'm not prepared to argue this. Again, sorry about that.

[YeGoblynQueenne]

>> Your calculator does have a representation of arithmetic too. It's those bits is moves around in registers, which are very much isomorphic to the relevant arithmetic.

Hm. Strictly speaking I believe my pocket calculator has an FPGA, a general-purpose architecture that in my calculator happens to be programmed for arithmetic, specifically. So I think it's accurate for me to say that, although the calculator has a program and that program certainly is a representation of arithmetic, I have to provide the interpretation of the program and reify the representation as arithmetic.

In other words, the program is a representation of arithmetic to me, not to the calculator. The calculator might as well be programmed to randomly beep, and it wouldn't have any way to know the difference.

(But that'd be a cruel thing to do to the poor calculator).