[Arjuna144]

I can highly recommend nobel laureate Sir Roger Penroses' take on AI. He argues that AI does in no way 'undestand' anything. He goes even further and uses Goedels Theorem to show that 'understanding' itself is non-computeable.

I am not sure about this, but after playing with chatGPT I can clearly see what he means by lack of 'understanding'. There is absolutely ZERO real understanding.... only correlations and probabilities of words....

[HarHarVeryFunny]

Why would you equate ChatGPT with AI ? It's just a language model. I know it's fashionable to call anything machine-learning related "AI", but it makes for non-sensical conversations such as here where Penrose is using "AI" to refer to any potential future AI (something that at least appears to have human level intelligence), but you're using "AI" to refer to something completely different - a dumb language model.

Penrose is one of those people who is "religiously" against AI. He will clutch at any straw possible to argue that AI will never be conscious, or understand anything, or encroach on our human uniqueness and/or spirituality in any way. His favorite argument seems to be that our brain is utilizing (inherently non-computable - at least by non-quantum computer) quantum effects in the brain's microtubules, but I'm sure if it wasn't that it'd be something else. He doesn't WANT to believe, therefore he finds an excuse not to believe.

[janalsncm]

Probably the main reason for this confusion is the lack of a good definition of “intelligence”. Basically no consensus if you browse the associated Wikipedia page. Of course the definition is also bogged down in human-centered history, assuming that humans are at the pinnacle of some contrived intelligence scale and placing other animals below us. I’m sure the genius philosophers who came up with this scale would’ve placed themselves at the pinnacle within humans as well.

Tacking “artificial” on top of that doesn’t make it any clearer.

My understanding of the taxonomy of these things within CS is that AI is a broad class of techniques for problem solving. Machine learning is a subset of those techniques which uses data and statistical methods. Non-ML AI is sometimes called GOFAI (“good old-fashioned AI”).

[HarHarVeryFunny]

AI and ML are two different things. AI can either be considered as a research area or simply end goal/result. AI doesn't itself refer to any set of techniques - it's the goal, not the means to the goal. You can consider GOFAI as non-ML, although perhaps it's more correct to regard it as older symbolic approaches (rule-based systems, etc) vs modern connectionist/ANN ones.

Machine Learning (ML) is a catch-all term (not in of itself a technology) for any approach where systems are designed to learn from data. This includes techniques such as random forests and SVMs, as well as neural nets.

There's really no fundamental relationship between AI and ML. Neither one is a subset of the other as this is an apples and oranges comparison - one is a goal, and the other is an approach). That said, all recent progress towards AI has been achieved by using ML, although not all uses of ML can really be regarded as AI.

Hope that helps define terms.

[janalsncm]

You may not see one as a subset of the other but there is no shortage of nested diagrams like [1], showing machine learning as a sub-field of artificial intelligence.

Further, the canonical Russell and Norvig AI textbook [2] only mentions machine learning briefly, as one of several skills that a computer would need in order to pass the Turing Test:

> machine learning to adapt to new circumstances and to draw new conclusions

And the Wikipedia page (which also contains a similar nested diagram) describes ML as a “part of” artificial intelligence.

So it is pretty clear to me that ML is treated as a sub-field of AI. However I still believe the AI field dances around the definition of Intelligence, preferring a practical task-oriented definition instead.

[1] https://www.edureka.co/blog/ai-vs-machine-learning-vs-deep-l...

[2] https://www.amazon.com/Artificial-Intelligence-Approach-Stua...

[3] https://en.wikipedia.org/wiki/Machine_learning

[HarHarVeryFunny]

Actually the WikiPedia page you linked to there shows two diagrams, one with ML as a subfield of AI, and another with them as separate overlapping fields (I guess the overlap is meant to represent where ML is being used to pursue AI).

> As of 2020, many sources continue to assert that ML remains a subfield of AI.[29][30][27] Others have the view that not all ML is part of AI, but only an 'intelligent subset' of ML should be considered AI.[5][31][32]

I'm well aware that there are some people who regard anything ML-related to be AI, which is the sensationalist view taken by the popular press who want to write stories about AI, but that doesn't mean it's true. What does something like a SVM classifier have to do with AI - it's more like a line fitting technique. Or what about a neural net that classifies photos as cat vs dog - seems better described as image recognition than AI (and the whole post-AlexNet modern neural net revolution was born out of the 2012 ImageNet image recognition competition). Or how about an echo-cancelling circuit in a telephone - is that AI to you? Why not - it's based on machine learning ...

As for ChatGPT, technically it's a language model, so really part of NLP - linguistics research. All it does it try to predict the next word in a word sequence. Of course it therefore ends up regurgitating lots of intelligent stuff it was trained on (as well as dumb and false stuff it was trained on), but that doesn't make it intelligent. Similarly, we don't say Google is AI just because it responds to search queries by finding web pages with intelligent content.

[nmfisher]

Just wanted to point out that ChatGPT is more than just a language model - from OpenAI's (very brief) description, it was also trained with reinforcement learning to select/rank the "best" answer [0].

I think the distinction is important because I suspect it explains why ChatGPT succeeds at certain tasks when previous LM-only models failed miserably.

[0] https://openai.com/blog/chatgpt/

[HarHarVeryFunny]

Yes, that's the difference between a plain language model like GPT-3 and a "task aligned" one like ChatGPT (which is based on GPT 3.5).

I'd describe it still a language model, but just one with "filtered" output.

I'm not sure if ChatGPT has been documented/described, but it's very similar to OpenAI's InstructGPT which they have described, and which they still refer to as a language model.

> We’ve trained language models that are much better at following user intentions than GPT-3 while also making them more truthful and less toxic, using techniques developed through our alignment research. These InstructGPT models, which are trained with humans in the loop, are now deployed as the default language models on our API.

https://openai.com/blog/instruction-following/

[bryan0]

What is understanding if not correlations of words and ideas and concepts? I totally respect Penrose but unless he has an noncomputable alternative for the brain then that’s just magic (and completely unnecessary). Now I know Penrose used to have ideas how the brain is actually a quantum computer which uses entanglement. But that is still computable.

[AbrahamParangi]

The bar should not be “does the machine understand” but rather, “can the machine be distinguished from a typical human by a test”

And further, for any test we choose we must ask ourselves, “do we expect our ability to discriminate between these two classes, man and machine, to go to zero as machine performance continues to improve”

If we believe that only continued performance improvements are needed to make AI indistinguishable then that means today we’re only talking about differences in quality rather than differences in kind.

[jcims]

User idiotsecant posted this excerpt from ChatGPT a couple of weeks ago. It kind of blows my mind (the last paragraph). It feels a bit beyond just basic probabilities of words.

https://news.ycombinator.com/item?id=33833178

We're just at the beginning.

[notahacker]

Didn't blow mine, even though it involved a howitzer! It's obviously operating on a higher level than "what was the previous word" but doesn't exactly feel like high level reasoning: when prompted for similarities it elides "inflated" with "blew up" to seed a paragraph (rather than to crack a corny joke, which is the only context I can imagine a human in possession of knowledge of what a howitzer and a beach ball doing likewise!), adds some sentences consistent with sentences in its corpus with "blowing up" and "loading" as the respective verbs and appends a sentence which states that it's tenuous, because howitzer and beach balls and inflating and loading verbs aren't often found in close proximity in its corpus (and caveating stuff clearly was well rewarded in ChatGPT's training; it's almost comically quick to do so in every single non-trivial answer)

Impressive that it does so in coherent English, but I'm not convinced it's in any danger of understanding firearms or fun beyond realising sentence forms those words would be a good fit and poor fit for.

I posted an excerpt from ChatGPT a couple of days ago where it stated - in an identical format to its previous two correct answers - that 355 was a prime number because it wasn't divisible 5, and that's the sort of problem that Turing machines are known to be able to completely and accurately model

[bratwurst3000]

My opinion is as long the dont solve hard problems they are just probability .

For example one of those top ten mathematical problems. Humans solved a few of those, lets challenge AI

[gnomewrecker]

By this metric the vast majority of humans are “just probability” too.

I think it’s not a very good metric.

[bratwurst3000]

Hmm is see it this way. Humans are a network of information sharing that could solve this problems. Computers are the same and should also Solve hard problems.

And yes, sadly, many humans are just probalility in my eyes.

[andyjohnson0]

Link?

> He argues that AI does in no way 'undestand' anything. He goes even further and uses Goedels Theorem to show that 'understanding' itself is non-computeable.

So how do brains understand things? Is he asserting that something is happening in the brain that can't be described/modelled computationally?

[astrange]

Penrose's argument is dualism (humans have souls) with a lot of mathematical handwaving around it involving claims there are quantum computers in your brain. I do not see why it should be respected or why you should think he's different from Deepak Chopra.

[anankaie]

Yes. Penrose hypothesizes a non-computational (which would be interesting, because it opens a whole an of worms about how to model it) constituent of the universe. A little while ago he, along with Stuart Hammeroff, proposed the Orch-OR model, which posited that this is the same constituent as what causes the "collapse" of an unmeasured superposition of states into one concrete, measured state. With that said, I have not seen any of his work on this for some time (I last looked at it around 2016), though a cursory look shows some new papers of his on this and related subjects.

[soulofmischief]

So in other words, humans also have zero real understanding.

[Jensson]

Humans add structure to unstructured data, the AI models we have just make flawed replications of structure we feed it, that is the difference. ChatGPT didn't even figure out the structure of basic math which is the most salient logical structure humans have. Even idiot humans can learn to count and compare quantities without being hardcoded to do so, they learn it from just words and pictures. A language model that failed to learn this when it was trained on the entire internet therefore can't have any understanding the way humans understands things, and feeding it more compute or data wont get it there either.

Example of a question a typical human idiot can solve without ever getting it wrong, but ChatGPT can't reliably: Is 7 dollars enough to buy a thing costing 7 dollars? ChatGPT can usually solve this, but it sometimes gets it wrong, getting that sort of thing wrong ever means that it doesn't understand, it just uses dumb statistics.

Edit: I'm not saying it is impossible to make AI models that understand these things, just that the ones we have today don't.

[chatgptexpert]

Dumb statistics you say?

https://i.imgur.com/im9EquK.png

Perhaps it's magic trick that a language model can do math, but it's quite impressive that it gets very close to the desired answer.

[Jensson]

It probably had problems with this part "two one dollar bills". A lot of text that contains "two" and "one" results in a "three", but also many texts with "two" and "one" results in "two", and then the model randomly chooses between those two interpretations. The worst part is that it doesn't even do it consistently for the same piece of text, creating that nonsense.

[jmfldn]

ChatGPT does not understand, I don't think there is any such thing as machine understanding nor will there ever be. At least not within the current paradigm of computing. My guess is that our kind of understanding requires a biological substrate and / or something quantum. Obviously I'm hand waving here as I'm out of my depth with this subject. My sense though is that there are things about the brain about its evolution that are eluding us, and which are significant.

Here's the thing though. Even on the current trajectory, we will produce programs that sure as hell APPEAR like they understand. This is really what Alan Turing was getting at when he said that the question of whether a machine can think is meaningless. It's like asking whether a submarine can swim. The fact is, dead machine or not, the outputs will increasingly exhibit, what look like, emergent properties of understanding, and even apparent sentience.

This to me is the truly stunning thing about ChatGPT. It might have limitations, but through mere statistics and 'auto complete on steroids' it can do way more than you'd think was possible with such an approach.

[Arjuna144]

True, except for this:

> Obviously I'm hand waving here as I'm out of my depth with this subject. My sense though is that there are things about the brain about its evolution that are eluding us, and which are significant.

Please go and deepen you meditations. First:

Get to the point, where you have stripped your awareness away from your sensory input (pratyahara)

(which everyone may know, it can be done, when you are deeply concentrated, someone may have said something to you, but you didn't "hear" it)

If you are at that point, your entire "experience" will consist of your "inner world" (thoughs and emotions). So next is to "concentrate" your mind (Dharana) by bringing it back to the same object (lets say a rose for example). This takes A LOT of practice but after the years, you mind will become clear and focused so that the time where only the object (the rose) is in your mind becomes longer and longer (a few seconds up to minutes).

As you practice further episodes of undisturbed, completely one-pointed mind become longer and longer and you enter into Dhyana (meditation). You are completely focused on the object, and the object itself is all that you experience. Imagine what that means: your entire experience constist of only the rose and nothing but the rose!!!

As this goes on and on for longer and longer periods of time, the yogis and mystics say that the boundaries between "you" the subject and "the rose" the object start to dissolve more and more, until they vanish. You are the rose and the rose is. Nothing but the rose that you are!! (They call it samadhi, the dissolution of individual existence and merging of object and subject).

This is just on an object but when the object is your awareness itself and this process happens, they call it many different names depending on tradition. ("Nirvikalpa Samadhi" in the yogic tradition, "Unio-Mystica" in the christian traditions, "fana" or "Baqaa billah" in the islamic mystical tradition. It is the realization of the unity of consciousness that is and that only is and that "I" merges into...

[astrange]

"Quantum" isn't magic - if you're a quantum computer, why can't you factor prime numbers in your head?

You're smarter than ChatGPT for a simple reason - you're not trapped in a computer and can decide what to spend time thinking about and where to go in real life to look up answers for your questions.

[hexatin]

Forgive me if this is covered in his argument since I haven’t read it yet, but does he discuss why this same argument wouldn’t apply to biological brains?

[eigenspace]

He thinks our biological brains get around it by using quantum computation.

Roger Penrose is a historical hero of the physics community and a current embarrassment from time to time. His ideas on things like computation and neurobiology are not worth taking seriously.

It’s unfortunate to see people like the one you’re responding to using his Nobel Laureate credentials to justify his nonsense.