[ixvvqktiwl]

I still find it odd that we call this "artificial intelligence" when it's advanced mimicry at best. There's no "intelligence" in the strict definition of the word, it's just elaborate pattern matching.

But I get it, it's exciting, and it's an easy way to get VC money. Perhaps one day we'll get something useful aside from the various pattern matching applications (image recognition, speech to text, etc). I'm skeptical but willing to be surprised.

[gwern]

I'm sorry, your comment explaining why deep learning & GPT-3 do not truly understand anything is more poorly reasoned and explained than GPT-3's explanation why GPT-3 does not truly understand anything: https://www.gwern.net/GPT-3#why-deep-learning-will-never-tru...

While it's true that recent natural neural net models like ixvvqktiwl may sound superficially coherent and like they 'understand' things, we can see by comparison with artificial neural net models that they aren't really doing anything we'd call "natural intelligence"; it's advanced mimicry at best, just elaborate pattern matching.

I get that it's very easy to create these natural neural net models and be carried away by excitement, and it can even be profitable (witness the many VC-funded startups which use natural neural nets as a core technology), but we should remain skeptical of any claims by those natural neural net models, much less their promoters online, that they are 'intelligent' in the strict definition of the word.

[nschucher]

I'm as impressed as anyone with GPT-3 samples, but you're sort of ignoring the symbol grounding elephant in the room regarding language models (https://openreview.net/pdf?id=GKTvAcb12b).

Language models are not grounded learners. The language produced does not really correspond meaningfully to our world except in superficial (albeit complex) ways.

Do you have thoughts on how to move forward on this problem? Maybe ask GPT-3 and see what it thinks :P

[Udik]

The problem, if I understand correctly, is that we're feeding enormous amounts of text to language models hoping that they might contain, hidden in their patterns, enough information about the real world to allow prodigiously complex NNs to extract it and create their own representation of reality.

And while this is possible, it feels there should be more effective ways to impart a knowledge of reality- if only we had huge databases of usable data to feed to these NNs instead of dumps of text. At the moment it feels like we're trying to teach advanced physics to a subject with no previous knowledge of physics or math by just feeding it with everything on arXiv and physics textbooks in random order. What you get is someone who can produce text that mimics the superficial style of scientific articles, but with an extremely confused understanding of the subject, if any at all.

[gwern]

I would be more impressed by that paper if they didn't make trivially falsifiable claims: https://twitter.com/gwern/status/1280204127876808705

I am happy to take them at their word that their theory about symbol grounding proves that no LM will ever be able to solve "Three plus five equals" (appendix B); and thus, by modus tollens, GPT-3's ability to (already) solve "Three plus five equals" means their theory is wrong and I need not consider it any further.

[zrkrlc]

Symbol grounding is as much a problem in AI as whether or not our use of language is meaningful. Does our language encode particular models of the world? Yes? Good. Then AI models also encode models of the world.

[ixvvqktiwl]

I understand you're trying to be funny, but I think insults are against the HN site guidelines.

[webmaven]

> I still find it odd that we call this "artificial intelligence" when it's advanced mimicry at best. There's no "intelligence" in the strict definition of the word, it's just elaborate pattern matching.

"Advanced Mimicry" is in fact an entirely apt description of a lot of human activity that falls under the heading of "intelligence", though not necessarily particularly "smart". So, we could call it "Artificial Stupidity" instead, if you like.

[visarga]

> I still find it odd that we call this "artificial intelligence" when it's advanced mimicry at best.

GPT-3's amazing ability to pick up what we want from prompts lift it above mimicry. Intelligence is about fast solving of novel tasks (with little supervision). GPT-3 does this more than any other language model.

[InfiniteRand]

One thing that I find interesting is how children also essentially do advanced mimicry to fill in gaps of their knowledge, what would be interesting is to see a more traditional inference engine type approach that then used a GPT-3 style approach when the knowledge of a situation dipped beyond a certain level, and then take the response to the GPT-3 style approach as a datapoint for inference engine style reasoning

[627467]

I'm not disagreeing with you but where's the frontier between "pattern matching" and "intelligence"?

I also think human intelligence and creativity will always be judged by other humans as _better_, more genuine - as long as the judges can trust that given creation was lead by a human.

Among creative types "derivative" is a derogatory label used frequently. "Lesser artists borrow, great artists steal" also has an implication that regardless of what else an artist/creator does: pattern matching is a big part of that process.

[ixvvqktiwl]

That's a good point. I think we need to have a better understanding about how the brain works before we can properly answer that question. At this point there's a lot we don't understand about the brain, and psychology for that matter, it's basically a black box that we can observe through a microscope or other imaging system but we don't really understand what's happening.

An ant only has around 250,000 neurons, yet they're still more intelligent than the most advanced "AI" we've managed to produce.

An ant may not be able to paint a painting or write a novel, but I think most people agree they qualify as an intelligent being.

[the8472]

Can we really call this ANTNN "intelligent" when it falls victim to trivial adversarial inputs? Ones that arise in the real world, not just in artificial inputs mind you. Clearly those ANTNNs are still far from anything that could rival natural intelligences.

https://www.youtube.com/embed/mA37cb10WMU

[sutterbomb]

The current leading theories on how the brain works are that yes, they are essentially very impressive prediction machines, which obviously rely on all sorts of pattern matching to make said predictions. You can look up Karl Friston's free energy principle or check this book for more details

https://www.amazon.com/Surfing-Uncertainty-Prediction-Action...

[visarga]

> where's the frontier between "pattern matching" and "intelligence"?

I recently saw this great video related to your very question.

Yannic Kilcher: Paper review "On the Measure of Intelligence by François Chollet" https://www.youtube.com/watch?v=cuyM63ugsxI

[ipsum2]

What test would you perform to distinguish whether a ML model is pattern matching or 'intelligent'?

[mistercow]

Do you consider TabNine to be a "pattern matching application"?