[SimianSci]

When someone is expected to be wizened and does not have the knowledge to keep up with the needs of those around them, they in turn become Shamanistic in their practice.

The speed of improvement on these models has been incredible and has outpaced the learning speed of humans and put many experts into these Shamanistic roles.

I think the operative means of addressing this is to recognize that we can only learn so quickly, but we are still called to improve our knowledge and understanding to a higher level. Since the improvement of these models is neither logorithmic, nor exponential, we currently occupy a space in time in which the models are currently smarter on average than we are as a collective whole.

[watutalkinbout]

Algorithms and data that emulate responses aren't smart.

A 5 year old knows if you want to wash your car, you need to take it to the car wash.

[coldtea]

Can a 5 year old write a substantial program on spec, that passes the requirements and given tests, in a few minutes?

If not, then perhaps this comparison is not the be all end all.

"A ship is useless, it can't drive over land..."

[dominotw]

5 year olds and ai both have jagged intelligence.

also its AI not "artificial code generation intelligence" . Ship is your view of the product to shoehorn into something specific.

[JBits]

But it demonstrates that LLMs struggle with basic reasoning. A criticism of LLMs is that they're imitating without a understanding of what they're doing and without a clear plan, so this inability to solve a simple logic puzzle is very relevant. If LLMs didn't struggle with reasoning problems then something like ARC-AGI wouldn't exist.

[Eisenstein]

Its a question designed to fool the AI. It's like saying that a person doesn't understand the limitations of reality when they fall for a magic trick.

[watutalkinbout]

These aren't the same thing.

You can't fool an AI because it isn't using its own judgement.

[Eisenstein]

You can fool an AI as evidenced by them being fooled. They demonstrably appear to be working through a problem and get fooled by the wording of the problem. If you think differently, merely asserting it is not the way to convince people that what they see is wrong.

[watutalkinbout]

You're describing a tool.

Tools can do very useful things, but they aren't intelligent.

[FarmerPotato]

referring to John Woolridge's recent talk? "the car wash is only 200 feet away, should I drive or walk?"

His slide showed Opus 4.6 saying "walk". I couldn't get 4.6 to do that.