[whiplash451]

I am not sure who the target audience of Gary Marcus is.

Those who know about LLMs are aware that they do not reason, but also know it not very useful to repeat it over and over again and focus on other aspects of research.

Those who don't know about LLMs simply learn to use them in a way that's useful in their life.

[marcosdumay]

I dunno. There have been 3 comments claiming they do reason on this page alone.

I doubt experts need to be reminded, but maybe non-experts need to see that incorrectness exposed, otherwise they'll get mislead.

[LgLasagnaModel]

Maybe his target audience is anyone who might have read (or written) this??? https://openai.com/index/introducing-openai-o1-preview/

- "A new series of reasoning models for solving hard problems. Available now." - "They can reason through complex tasks and solve harder problems than previous models in science, coding, and math." - "In a qualifying exam for the International Mathematics Olympiad (IMO), GPT-4o correctly solved only 13% of problems, while the reasoning model scored 83%." - "But for complex reasoning tasks this is a significant advancement and represents a new level of AI capability." - "As part of developing these new models, we have come up with a new safety training approach that harnesses their reasoning capabilities to make them adhere to safety and alignment guidelines. By being able to reason about our safety rules in context, it can apply them more effectively. " - "These enhanced reasoning capabilities may be particularly useful if you’re tackling complex problems in science, coding, math, and similar fields."

there are a few more in that post, but clearly OpenAI is pushing the reasoning thing A LOT

[shprd]

If only that was the case. Laywers used them in courts, key people are using it to analyze reports and make decisions for them because it's "AI" and advertised as better than humans. The problem is LLMs output look coherent and make sense so with the advertising, people are misled about what it does and what is capable of.

People are only hearing about AI, how it's revolutionary, and how it's master in every field.

It can solve questions better than me so why would I not use it to help me with everything that I can't figure out?

There are billions spent in marketing to make people buy these products. No one is telling customers to figure it out and see if it's useful.

Even many technical people started getting lost:

  you know what? maybe it does reason. I asked it this novel trick question and it answered correctly. This is a new model, we don't fully understand its capabilities yet.

You might be able to spot little "mistakes" and "exaggeration" and see they're just selling it but people accumulate those "exaggeration" from here and there and build on them collectively.

[glomgril]

He is coming from the perspective of a long-running debate on symbolic versus statistical/data-driven approaches to modeling language structure and use. It seems in recent years he has had trouble coming to terms with the fact that at least for real-world applications of language technology, the statistical approach has simply won the war (or at worst, forms the core foundation on top of which symbolic approaches can have some utility).

I come from the same academic tradition, and have colleagues in common with him. He has been advocating for a quasi-chomskyan perspective on language science for many years -- as have many others working at the intersection of linguistics and psychology/cog sci.

TBH I suspect he himself is a large part of his target audience. A lot of older school academics raised in the symbolic tradition are pretty unsettled by the incredible achievements of the data-driven approach.

Personally I saw the writing on the wall years ago and have transitioned to working in statistical NLP (or "AI" I suppose). Feeling pretty good about that decision these days.

FWIW I do think symbolic approaches will start to shine in the next several years, as a way to control the behavior of modern statistical LMs. But doubtful they will ever produce anything comparable to current systems without a strong base model trained on troves of data.

edit: Worth noting that Marcus has produced plenty of high-quality research in his career. I think his main problem here is that he seems to believe that AI systems should function analogously to how human language/cognition functions. But from an engineering/product perspective, how a system works is just not that important compared to how well it works. There's probably a performance ceiling for purely statistical models, and it seems likely that some form of symbolic machinery can raise that ceiling a bit. Techniques that work will eventually make their way into products, no matter which intellectual tradition they come from. But framing things in this way is just not his style.