[sixQuarks]

Elon Musk disagrees with that statement. He believes once intelligence gets beyond a certain threshold (which humans have), there are no such things that are beyond understanding. Most people don't know how to write complex math equations, but that doesn't mean they can't understand what mathematics is.

[unclebucknasty]

In discussions about artificial super-intelligence (the final stage of AI that sends us rocketing beyond the singularity), our inability to comprehend what such an intelligence might desire or do is frequently accepted as a significant risk by those who fear such AI.

When one considers an infinitely and exponentially increasing intelligence that can quickly bypass the point of easily answering the questions that currently most perplex us in all fields, then it is reasonable to believe that intelligence would arrive at a level of considering (and solving) problems that we cannot even imagine, let alone understand.

Given that Elon Musk has spoken about has concern over AI, it would be interesting to hear how he integrates that with his belief that our intelligence is sufficient to understand anything.

[eridius]

We may not be able to predict what such an AI would desire or do, but I assume you'll agree that we'll still be able to observe what it does do and understand that it's doing it (even though we may not necessarily understand how it does something).

[unclebucknasty]

>I assume you'll agree that we'll still be able to observe what it does do and understand that it's doing it

No. To make that leap, I must assume the conclusion. I cannot know that I'll be able to observe what it does, because I don't know what it will do (or whether I can even perceive the dimension in which it does it for that matter).

Secondly, the word "understand" is subjective. I assume that Musk meant the word in the sense that there is sufficient depth of understanding as to actually be meaningful, else his statement itself would be meaningless (i.e. so obvious as not to be required).

[eridius]

I had a really long reply written, but before posting I went to try and find the source of the Elon Musk paraphrased belief, and I can't find it. I don't know where sixQuarks got it from, but without knowing what the source is, it's hard to second-guess what Elon Musk was actually trying to say (or whether he even said it at all).

That said, ignoring the Elon Musk bit, even if a superintelligent A.I. figures out how to communicate in some other dimension, surely there's plenty of work it will do that fits within the bounds of the physics we know, and we can observe what it does that we do understand, and we can observe what it does that we don't understand. And given time we can figure out how to understand that which we can observe. And more generally, even if it manages to jump straight to communicating in some other dimension without letting us see its intermediate work, it seems very likely that whatever it is doing will have observable effects within the realm of the physics we understand and can observe (even if it's just EM radiation). At the very least, we can tell that it's there and that it's doing something and start figuring out what it is doing.

This is in contrast to the ants of Michio Kaku, who don't even know anything is going on. The point isn't that the ants don't understand how to build their own 10-lane superhighway, it's that they don't even know that this is a thing built by another species or for what purpose it was built. sixQuarks's point (regardless of whether what he said about Elon Musk is true), summed up in his analogy about complex mathematics, is that we should still be able to observe aliens and comprehend that they're doing something, even if we don't necessarily understand how to do that thing ourselves.

[sixQuarks]

Here is the source: http://waitbutwhy.com/2015/05/elon-musk-the-worlds-raddest-m...

And here is the actual paragraph:

"One topic I disagreed with him on is the nature of consciousness. I think of consciousness as a smooth spectrum. To me, what we experience as consciousness is just what it feels like to be human-level intelligent. We’re smarter, and “more conscious” than an ape, who is more conscious than a chicken, etc. And an alien much smarter than us would be to us as we are to an ape (or an ant) in every way. We talked about this, and Musk seemed convinced that human-level consciousness is a black-and-white thing—that it’s like a switch that flips on at some point in the evolutionary process and that no other animals share. He doesn’t buy the “ants : humans :: humans : [a much smarter extra-terrestrial]” thing, believing that humans are weak computers and that something smarter than humans would just be a stronger computer, not something so beyond us we couldn’t even fathom its existence."

[eridius]

Thanks. It looks like that's a lot more applicable to the Michio Kaku scenario than it is to superintelligent A.I., because the question is whether we can comprehend the existence of aliens, not whether we can understand the technology that a superintelligent A.I. might create, or even understand its motives (comprehending the existence of some other intelligent entity is not the same as understanding what the entity wants or why it does what it does).

[unclebucknasty]

We may be talking past each other here and I'm not quite sure how to formulate my thoughts more clearly, but I'll try.

>surely there's plenty of work it will do that fits within the bounds of the physics we know

Simply to say some work would itself be a huge assumption, but plenty? That assumes far too much about our current state of knowledge, and it places artificial limitations on the capacity of a super-intelligence.

The flaw in your thinking is your very reliance on our own knowledge as your frame of reference. You're extrapolating from that because you can't imagine anything else. But, that's the point: we don't know what we don't know (or what a super-intelligence might know).

EDIT: I could attempt to use hypothetical scenarios here to convey how your assumptions might be wrong (e.g. what if this intelligence could jump dimensions such that our laws of physics no longer hold?). However, to do that would be to commit the same error that you're committing: it extrapolates back from our own limited language and understanding. By definition, if we can explain it, then it doesn't meet the test of the type of intelligence that I'm referencing./EDIT

Your statement is also a pretty big tell that you're working from an assumption of only marginally increased intelligence, which is where I think our disconnect really enters. To appreciate the degree of intelligence I'm referencing, see [1].

It's long, but a good read in its own right. Specifically regarding this discussion, towards the middle of the page there are two graphs, labeled "Our Distorted View of Intelligence" and "Reality" which might help illustrate the scale of what I'm trying to communicate. Here also is an attendant quote:

>In our world, smart means a 130 IQ and stupid means an 85 IQ—we don’t have a word for an IQ of 12,952

See what I mean? When intelligence reaches such a scale beyond our own, it destroys key assumptions. The idea that we can comprehend that the intelligence is doing something is not guaranteed. Further, even if perceived, that perception itself is potentially only incrementally more representative of true understanding than not perceiving it at all. So, for any meaningful purpose, our understanding would be much more closely aligned to that of the ants WRT the 10-lane superhighway.

Another way to look at this is to imagine an infinitely expanding intelligence scale. At some zoom-level, we are pushed so much closer to the ants that the difference is virtually indiscernible.

[1] http://waitbutwhy.com/2015/01/artificial-intelligence-revolu...

[sumitviii]

I was going to make logingone's argument but here's a better one:

All maths done till ~1600 AD could have been done in 50 years, if you see it from perspective of a civilization which has a stable existence. And forgetting all abstractions, stability of surrounding civilizations is basically a random phenomenon.

And I'm sure maths done in physics departments is beyond understanding of 80% people outside it, let alone any civilization which started 400 years before us.

[tripzilch]

Okay, but there's still a difference between:

1. Knowledge that is potentially within our understanding, as humans, even if it takes a few centuries.

2. Knowledge that would require a few millions of years, because we'd need to evolve beyond our current level of intelligence, similar to what chimps, as a species, would need to do if they were to really understand theoretical physics (that is, assuming chimps would in fact evolve to higher intelligence, if there were, say no humans around).

A reasonable interpretation of Elon Musk's claim would be mainly arguing against the existence of a barrier like the type 2 knowledge. But I could be entirely wrong, I haven't read his reasoning behind this claim.

[sumitviii]

How do you think Newton would have reacted if someone said "Pentaquark" in front of him? Think he would have understood when just the word was said? Musk is talking about Newton getting all of modern physics when all he has heard is two undergrads chat on Hacker News.

[logingone]

Elon Musk is an engineer, not a philosopher, and I stand to be persuaded, but I find using him to counter Michio Kaku ridiculous.

[adventured]

A philosopher would be the first to understand that no one is or can be excluded from the realm of ideas or thought.

Being a philosopher is not an extraordinary designation, such that it puts them beyond the mental abilities of someone of Elon's obvious mental aptitude.

[logingone]

Exclusion? Oh dear, here we go with the black or white. "realm of ideas or thought" - and the simplification. "not an extraordinary designation" - and dismissing philosophy. "beyond the mental abilities" - and more simplification. "mental aptitude" - you'll be fine then with neuroscientists or some such fixing the falcon.

[pekk]

It really doesn't matter who said it, it's just good to cite the source. But there is nothing which makes Michio Kaku infallible. Arguments can't be reliably evaluated according to who made them.

[sixQuarks]

Like him or not, Elon Musk has an uncanny ability to figure things out. Of course he's wrong on some things, but for the most part, I wouldn't discount anything he says.

[ekianjo]

> doesn't mean they can't understand what mathematics is

Yet humans clearly fail to understand things like radioactivity (and the risks involved, for good or bad). I'm not sure what Musk thinks is reliable at all in this matter.

[tosseraccount]

Humans do understand radiation dangers. Why do we have no nuclear war since the atom bomb? Why are there so few deaths from nuclear power plants? Why do people use sunscreen and shield spacecraft?