[frabcus]

Roger Penrose basically suggests what you say in "The Emperor's New Mind". Roughly, it says that the brain (likely, according to him) uses quantum computation, and so we can't make an AI out of a classical computer.

The practical flaw with this argument, of course, is that you could instead make an AI that itself uses quantum computation. I asked Roger Penrose about this at a university philosophy meetup over 20 years ago, and he agreed.

Likewise, if there is some kind of halting oracle, perhaps we can work out how the brain creates and connects to that oracle, and make our AI do the same.

Meanwhile, there is no physiological or computational evidence for this possibility. We should keep hunting though, as that's the same thing as understanding the detail of how the brain works!

[yters]

Well, quantum computation is weaker than a nondeterministic Turing machine, so not the same thing I'm saying. Penrose correctly identifies the mind cannot be a deterministic Turing machine, but his invocation of quantum mechanics does not solve the problem he points out. A DTM can simulate an NTM and hence anything inbetween, so the inbetween of quantum computation does not solve anything.

The fundamental problem Penrose identifies boils down to the halting problem, which requires a halting oracle to be solved. Hence, a halting oracle is the best explanation for the human mind, and no form of computation, quantum or otherwise, suffices.

UPDATE:

Since I'm rate limited, here is my answer to the replier's comment:

A partial answer: the mind has access to the concept of infinity, and can identify new, consistent axioms. Other possibilities: future causality and ability to change the fundamental probability distribution.

But, it's also important to note that we don't have to answer the "how" question in order to identify halting oracles as a viable explanation. We often identify new phenomena and anomalies without being able to explain them, so the identification is a first step.

[sjeohp]

>But, it's also important to note that we don't have to answer the "how" question in order to identify halting oracles as a viable explanation. We often identify new phenomena and anomalies without being able to explain them, so the identification is a first step.

I don't think it constitutes an explanation at all, let alone a viable one, if all it does is beg the same question.

The problem was already identified: "how does human cognition work?" You've renamed it: "how does this supposed halting oracle work?" That might be an interesting framing but it is not a viable explanation of anything until you've proved that such oracles exist or in other words, solved the halting problem.

[sjeohp]

>Hence, a halting oracle is the best explanation for the human mind

What does it explain though? That the human brain has a black box capable of solving certain problems... how exactly?

[mannykannot]

Indeed - it's essentially the homunculus fallacy, or magic dressed up in the language of knowledge.

https://en.wikipedia.org/wiki/Homunculus_argument

[brokensegue]

Your theory doesn't seem falsifiable short of actually making an agi whose very definition is notoriously slippery.

You might as well just say the mind resides in the soul.