[sarchertech]

Penrose doesn’t hold the microtuble hypothesis strongly at all.

He’s very very careful to say that it’s just something he’d like to see tested and he has no idea whether it’s true or not.

That very much distinguishes it from Crank science.

[btilly]

In 1989, Penrose picked up Lucas' 1961 argument that no computer can possibly simulate intelligence. The argument rests on fundamental misunderstandings of logic, that are well-known among logicians. See, for example, https://www.ams.org/journals/bull/1995-32-03/S0273-0979-1995... for an article explaining this, written some 30 years ago.

The fact that Penrose has maintained his misunderstandings for 30 years, demonstrates that, on this topic, he has been a crank for a long time. No matter his other accomplishments.

[sarchertech]

Penrose essentially wrote an entire book responding to that critique.

Which side you support largely comes down to a philosophical question. The notion that he just made a stupid mistake and doubled down on it is absurd.

Having a different opinion about the soundness of a proof could make him wrong but it hardly makes him a crank.

[btilly]

The fact that he repeated the same logic errors at book length doesn't change the fact that they are errors. And the question of whether they are logic errors is a question of mathematics, not philosophy. Dismissing the conclusions of logic on the basis of philosophy, is a mistake of the same type as dismissing the conclusions of science on the basis of theology. Logic cannot speak to the philosophy that Penrose pushes forth. But it can and does speak to the validity of the argument from logic that he puts forth in support.

Hilary Putnam did a good job of explaining the mistakes. I am not a logician, but my background in logic is good enough to verify the explanation. And every logician that I personally know has come to the same conclusion.

Like you, I find it absurd to claim that Penrose has been doubling down on a basic logic error. And yet we have the basic logic error, and Penrose has clearly been doubling down on it.

You don't even need to be an expert to understand that he can't be right. Penrose argues that the capacities of human reasoning is such that Gödel's theorem proves that a mathematician's brain cannot be replicated by any mechanical process. But the reasoning process that mathematicians use is fallible. The output most emphatically is not logically consistent. The appearance of consistency is only obtained after much reexamination of those errors which were discovered. Absolute certainty of lack of error is unachievable by any kind of human reasoning. The history of mathematics is filled with examples of errors that were not discovered for shockingly long periods.

So we do not have a proof of the consistency of human reasoning, or its products. Therefore Gödel doesn't apply. Human reasoning, including the outputs that Penrose cites, do not strictly follow first order logic. Therefore Gödel again doesn't apply. And Gödel is entirely silent on the potential prospects of a heuristic algorithm that can produce inconsistent results. Which is what our brains do.

The inapplicability of Gödel's theorem to our thinking process is an absolute barrier to Penrose's attempts to prove that our thinking process cannot be the result of a mechanical system. It may be that it is not. Personally I fail to see how a strictly mechanical process can create my experience of consciousness. But this is a question that Gödel's theorem cannot address.

[sarchertech]

The mistake you are making is the same that I think a lot of people who dismiss his argument make.

You aren’t reading his actual argument. You are reading a characterization of his argument by a critic.

You can read his rebuttal of those critiques here:

https://calculemus.org/MathUniversalis/NS/10/01penrose.html

The summary is that there is no requirement that human reasoning is infallible in his actual argument.

Again his proof may be faulty. But it is not because of a “basic logic error”. The disagreements people have with his actual argument are much much subtler than a basic logic error.

[btilly]

The mistake that you are making is to imagine that I must be making a mistake.

Let's take a few examples.

He claims that a robot which is able to engage in Gödelian reasoning, cannot possibly be computable. Logicians agree that this claim is false. Indeed https://link.springer.com/article/10.1007/s10817-021-09599-8 shows a version of Gödel's theorem that has been fully checked via proof assistant. While we still lack AIs that are able to produce such proofs (other than by regurgitating such proofs in their input data), in principle a proof checker filtering the output of a brute force search through possible proof attempts will achieve any possible machine checked proofs. But proof checkers can proof check Gödelian reasoning. Thus we already know how to write a (rather impracticable) robot that does exactly what Penrose claims to be impossible.

Here's a whopper. Let's go to this passage from 4.2 of his rebuttal.

However, I had been disturbed by the possibility that there might be true mathematical propositions that were in principle inaccessible to human reason. Upon learning the true form of Gödel's theorem (in the way that Steen presented it), I was enormously gratified to hear that it asserted no such thing; for it established, instead, that the powers of human reason could not be limited to any accepted preassigned system of formalized rules. What Gödel showed was how to transcend any such system of rules, so long as those rules could themselves be trusted.

This is complete and utter bullshit. Gödel did not show that we could transcend any such system of rules. What Gödel demonstrated is what those rules can prove of themselves. Namely, "If this set of axioms proves itself consistent, then it is inconsistent." Which statement can be proven using nothing more than arithmetic. Our ability to prove this doesn't prove that our mathematical reasoning is somehow beyond what a mere formal system can prove. It is just a demonstration that we can follow a piece of arithmetic to its logical conclusion. Any other understanding of the result is simply a mistake.

He's also wrong about whether there are problems that, in principle, are beyond human reason. For example consider the BB(n) problem. Identifying which Turing machine gives us BB(643) is impossible from ZFC. (See https://github.com/CatsAreFluffy/metamath-turing-machines for more.) If you go to BB(1000), no set of axioms that mathematics has ever debated can suffice. Going beyond human comprehension doesn't take much more than that.

Of course those are weak estimates. In fact it is likely that BB(10) is going to be forever beyond us. And no, some magic quantum decoherence in the microtubules isn't going to fix that.

Let's move on. Section 4.5. He admits to the logical possibility that he is wrong, then asks whether unsoundness is plausible. How is it not plausible? The only form of intelligence that we have an existence proof for, us, thinks in notoriously unreliable ways. LLMs are our best attempt to replicate our verbal abilities by computers. They are likewise extremely unsound.

The burden of proof that soundness is possible here is on Penrose. And he needs to prove it soundly enough to overturn the generally accepted conclusion that the known laws of physics suffices, in principle, to explain the manner by which our brains operate. Because that is the conclusion that he is aiming to convince people of.

He doesn't even try. He waves his hands, declares absurdity, and moves on. That may be fine from the point of view of his philosophy. It is not fine from the point of view of a logician. It's a gap. And a mighty big one at that.

I could go on, but what's the point? If you refuse to believe what logicians say about logic, then no explanation of what logicians have to say will convince you. And if you do believe what logicians say about logic, then you should already know that Penrose is wrong.

[sarchertech]

>Humans are unsound, so Gödel doesn’t apply.

As Penrose points out, the claim is conditional. Take a recursively axiomatized theory T for which we have Pi_1 soundness level warrant, i.e., we have reason to think its Pi_1 theorems are true in the standard model N for the arithmetical cases we care about. Then we can see (again: in N) that G_T is true while T cannot prove it.

That gives principled grounds to adopt Pi_1 reflection or otherwise step to a stronger T'. No infallibility claim about people is require. This mirrors the ordinary kind of warrant mathematicians use when they adopt new axioms after scrutiny and debate.

>A proof checker plus brute-force search can do Gödel reasoning, so a robot can do it.

A checker plus search enumerates exactly the theorems of whatever fixed system it’s tied to. You can also script computable progressions that iterate reflection or consistency along recursive ordinal notations in the Turing/Feferman style. That’s still a single computable progression determined in advance. It isn’t that such progressions don’t exist, but that our justified acceptance is not a priori bounded by any one fixed computable progression.

The mechanist reply here is an existence thesis: there exists some computable procedure whose output matches everything humans could in principle come to rightly endorse for arithmetic. If that’s your view, give the existence argument. If instead you propose a specific computable progression P that we could in principle ratify as Pi_1 sound in toto, Gödel/reflection immediately pushes past P. If you say we can’t justifiably ratify P as a whole, you’ve conceded the point that no single precommitted computable scheme captures the moving boundary of what we’re warranted to accept.

>Gödel didn’t show transcendence.

In the narrow sense above, Gödel shows how to go beyond any accepted, fixed rule set once we have Pi_1 soundness level warrant for it. That is exactly the move at issue. Note the scope: inside a system you can’t adopt “if provable then true” without collapse by Löb. Outside, adopting restricted Pi_1 reflection is the justified step. If you want to reject the move, reject the warrant. Deny that we sometimes justifiably regard a given fragment of T as Pi_1 sound for the cases at hand. That’s an objection about epistemic warrant, not a logic gotcha.

>Busy Beaver shows limits, so case closed.

Busy Beaver actually helps here. It yields concrete Sigma_1 statements that can be independent of strong base theories, illustrating that warrant-driven extensions are sometimes needed to settle specific instances.

Either sometimes we do have adequate Pi_1 level meta warrant for a theory T on the arithmetical claims we care about. In which case we can recognize G_T as true in N and rightly move to T', ensuring our recognitions outrun that fixed T. Or deny such warrant altogether, in which case you haven’t refuted the conditional. You’ve just changed the target to a weaker notion of “what we can recognize.” The “humans are unsound” line doesn’t touch that conditional, and “just use a checker plus search” doesn’t answer the challenge unless you can support the existence of a single computable theory or precommitted progression that captures everything we could in principle come to rightly accept for arithmetic.

[spot]

yup and his book was reviewed as such at the time. mention of the rejection of his theory by professional philosophers however keeps getting edited out of the wikipedia page. See this exchange on the talk page https://en.wikipedia.org/wiki/Talk:The_Emperor%27s_New_Mind

>> "The book's thesis is considered erroneous by experts in the fields of philosophy, computer science, and robotics."

> Wooooah, there. That's a massive accusation to add, unsourced, and without any discussion. There needs to be a source for this statement, not to mention an opposing view. It seems unlikely the guy would win an award for a book no one thinks is right. I'm deleting it unless someone comes up with a pretty good source. Joker1189 (talk) 20:43, 27 July 2010 (UTC)

The source was provided with the edit: L.J.Landau (1997) "Penrose's Philosophical Error" ISBN 3-540-76163-2 http://www.mth.kcl.ac.uk/~llandau/Homepage/Math/penrose.html Spot (talk) 03:08, 31 July 2010 (UTC)

[sarchertech]

You say keeps getting edited out but I see one instance of that happening 15 years ago.

The talk is unavailable, but that and a chapter in Landau’s book hardly seem like appropriate sources for “everyone disagrees with him”.

The vast majority of critiques of his argument that I’ve read are by people who are actively working in AI not professional logicians.

[spot]

who said "everyone" or did you make a strawman?

https://en.wikipedia.org/wiki/Penrose%E2%80%93Lucas_argument

this actually puts it better:

The Penrose–Lucas argument about the implications of Gödel's incompleteness theorem for computational theories of human intelligence was criticized by mathematicians,[16][17][18][19] computer scientists,[20] and philosophers,[21][22][23][24][25] and the consensus among experts[7] in these fields is that the argument fails,[26][27][28] with different authors attacking different aspects of the argument.[28][29]

so, rejected by consensus. someone should update the book page so this expert rejection is clearer.

[sarchertech]

Everyone is obvious hyperbole because I didn’t want to both copying and pasting the actual quote.

That is merely a compiled list of people who disagree with him without listing any of his supporters.

There are at least 5 philosophers who support his position if you follow those links and 5 who reject it.

Link 7 doesn’t support the statement “consensus among experts in these fields” because it only refers to a single field—philosophy.

Many of those sources are just links to lists that other people have compiled of arguments for and against Lucas’ argument. They aren’t even all critiques. And many of the ones that are, are already linked directly in the article.

There’s is nothing more to support the notion that there is widespread consensus against his argument.

There may be. But this isn’t good evidence of it.

[btilly]

I can't speak to the general opinion among philosophers about his argument. But my opinion about philosophy is such that their opinion would not sway mine in either way.

I can speak to the general opinion of his logical arguments among logicians. And it is not just widespread consensus against. It is a widespread consensus that the argument is filled with basic logic errors that render it absolutely wrong.

As Hilary Putnam points out, Penrose's arguments are even worse than Lucas'. In particularly Penrose argues that no program that we can know to be sound, can simulate all our human mathematical competence. But our brains do not use a sound thinking process. Therefore a sufficiently good simulation of our brains that it can do mathematics, would also not be sound. Gödel's theorem is entirely silent on the potential capabilities of such unsound systems.

Furthermore LLMs provide a convincing demonstration that unsound simulations of us can have surprising levels of competence. ChatGPT regularly demonstrates both its competence and unsoundness. Sometimes at the same time!

The potential for unsound systems to demonstrate competence far beyond what most expected, is demonstrated by LLMs. Admittedly the current error rate is unacceptably high. But it demonstrates that what Penrose claimed to be mathematically impossible, may plausibly become real within our lifetimes. (Though, given how old Penrose is, not his.)

[russdill]

That's the problem with Penrose's thinking though. He's absolutely convinced that consciousness cannot boil down to something computable. So he reaches for the quantum shelf, but not just the quantum shelf, the quantum processes we don't yet understand since otherwise it'd just be something computable, but with more steps.

[sarchertech]

I mean most AI researchers are utterly convinced that the human brain is a Turing machine. There’s not reason to presuppose that. And if everyone presupposes that and it isn’t true, we’ll be stuck spinning our wheels until someone questions it.

[russdill]

This isn't a viewpoint coming from AI, is been a view that's been around for quite some time. The standard model does an exquisite job of predicting the day to day physics we experience. To find search for new physics where we might find discrepancies, we have to build absolutely huge colliders, and even those don't really effect our "day to day" physics.

For consciousness to be based on some non computable function there would have to be some unknown physics occurring in the brain. Sure, that's hard to disprove, but it also strains credulity, hard.

Penrose picking "quantum" for being the element of physics that would have to change to allow this non computibility is just woo. Why not just say magic?

[pmontra]

I agree with you that picking quantum was probably the fad of the time and it was an easy pick as quantum is everywhere. He cold have said "electrons". However the argument against unknown physics is not very sound IMHO.

There has been unknown physics at play inside brains since forever and it still is and always will be, by definition of science.

The point is that we don't even know how to define consciousness and humanity doesn't have a shared agreement about which living beings are conscious or are not. We're still like engineers building things millennia or centuries ago with only a shadow of a theory of why their creations worked. And yet we still walk on bridges from 2000 years ago and we had electric batteries and power plants before knowing how an electro magnetic wave moved.

[russdill]

If it were known that there where physical interactions occurring within the brain that deviated from the standard model, we wouldn't bother building didn't colliders.

[sarchertech]

Even if the brain showed a real deviation from the Standard Model, we’d still need colliders. The brain is a messy, low energy environment, and observation isn’t identification.

Colliders provide clean, high energy tests to pin down masses, spins, and couplings, and we already need them for open problems. A bio anomaly would set targets, not replace colliders.

[sarchertech]

> This isn't a viewpoint coming from AI

It has always been related to AI. Shortly after Church and Turing formalized computability, people started squaring off into 2 camps. People who believed strong AI was possible and those who didn’t.

We know the standard model is incomplete. Penrose’s ideas come directly from his his explorations of the gaps he suspects exist.

[russdill]

There are people that come to it from the direction of AI because they think they know something about consciousness. That's certainly putting the conclusion first.

But there are also physicists who come at the problem from the direction of physics. We know the standard model is incomplete, but we also know it covers everyone we experience with exquisite precision. Unless there is a black hole or temperatures on par with the big bang going on within our brain, the standard model will tell you what you need to know

[sarchertech]

You are vastly overstating the certainty of your position. Many physicists, including Penrose (even if ignoring his consciousness arguments) are deeply unsatisfied with the standard model and believe that it is incomplete. Despite understanding how useful it can be.

It is very unlikely that it can tell you “everything you need to know” or even everything that could possibly be useful.

Even if that were the case we have no way of knowing that or even supporting that with our current understanding. That takes that statement thoroughly outside of the realm of science and into philosophy.

>Unless there is a black hole or temperatures on par with the big bang going on within our brain, the standard model will tell you what you need to know

Do you think that if it were that simple, that a world renowned physicist would capable of doing the back of the envelope math?

Also do you think that if it were that simple that this world renowned physicist could consistently convince other world renowned physicists to engage with him beyond simple 1 paragraph rebuttals? To the point where they will write entire chapters in books published by him?

You may not find his argument convincing, but his arguments aren’t dismissed as crank science outside of edgy internet posts.