[fmbb]

> It’s because the bottleneck isn’t in intelligence, but in human tasks: specifying intent and context engineering.

So the bottleneck is intelligence.

Junior engineers are intelligent enough to understand when they don't understand. They interrogate the intent and context of the tasks they are given. This is intelligence.

Solving math questions is not intelligence, computers have been better than humans at that for like 100 years, as long as you first do the intelligent part as a human: specifying the task formally.

Now we just have computer programs with another kind of input in natural language, and which require dozens of gigabytes of video ram and millions of cores to execute. And we still have to have humans to the intelligent part, figure out how to describe the problem so the dumb but very very fast machine can answer the question.

[agentcoops]

I'm not sure your argument applies only to AI. Intelligence is certainly not knowing through, say, divine inspiration what another person wants you to do. This bottleneck of "describing the problem" is the same bottleneck faced when working with junior (or senior) engineers, especially in a team. One need only consider the classic of our field, Mythical Man-Month, which is really dedicated to this precise and, in some sense, irresolvable problem -- often it's best to just have one person who understands and ideally first posed the problem do the work, rather than introduce this bottleneck of communication.

It's a difficult and crucial problem, we all agree, but it's a stretch to define intelligence as such to be "describing the problem." Choosing the right problem in the first place (i.e. not just telling person B to do X but selecting the X that in fact is worth pursuing), perhaps, but I don't think that's right either as a definition of intelligence. Indeed, even the best scientists often speak of an "intuition" that drives their choice of problems.

More classical definitions place intelligence in the domain of "means-ends rationality", i.e. given an end to pursue being capable of determining the correct way to do so and carrying it out until completion. A calculator like a hammer is certainly not intelligent in that sense, but I would struggle to see how even an AI skeptic could maintain that state-of-the-art LLMs today are not a qualitative step above calculators according to this measure.

[fmbb]

All living things have means and ends and pursue goals to completion. That does not make us call them intelligent.

Whenever the LLM fails to act intelligently, we blame the person who gave it the task. So we don't expect them to be able to figure anything out, we are just treating them as easily reconfigurable Skinner boxes.

I'm not an expert or even very interested in the field so I cannot judge what you propose, only intuit from the word "intelligence" and how these machines are described to work and how I observe them working. Reading a bit of https://en.wikipedia.org/wiki/Intelligence leads me to believe these machines have even less to do with any classical definition of intelligence, but I did notice that

> Scholars studying artificial intelligence have proposed definitions of intelligence that include the intelligence demonstrated by machines

which seems rather relevant. Yeah when the AI researchers describe intelligence the machines are intelligent.

[pvtmert]

I truly love this comment, which essentially says: LLMs are glorified calculators, with ambiguous grammar. :)

[bamboozled]

That’s really what they feel like to me, a type of word / number hybrid calculator. Like a probability machine…you attempt to give it the right input and you hopefully and non demonically get some output.

[handzhiev]

Computers are glorified calculators, yet they power most of our lives

[giantrobot]

Many computers and interfaces are deterministic. LLMs are by nature not deterministic and not even non-deterministic the same way on any two invocations given the same prompt and context. Natural language is ambiguous and for many languages very context dependent. It's not the greatest interface for a calculator from which we're expecting deterministic accurate answers.

WolframAlpha is a more impressive front end to a calculator than I've seen out of LLMs. Not only does it show me how it translated my natural-ish language query but it shows me potential alternative interpretations to my question. LLMs by the nature of how training works can't necessarily tell me why and how they interpreted my prompt. The thinking models are better but still not great.

[exe34]

> Junior engineers are intelligent enough to understand when they don't understand. They interrogate the intent and context of the tasks they are given

Eh, I wouldn't apply that as if it's a general thing. yes, the really good ones do. many will equally plough through into the mud with albeit admirable determination.

[fmbb]

yeah so the LLMs are like a stupid junior

only they work for free and produce megabytes of stupid code per hour

[exe34]

a junior that can instantly fix a problem spotted by a senior even if it involves touching 10 files. no meat junior can do that.

[elevatortrim]

Yes but for the purposes of these conversations, we do not need to say "good juniors" "good engineers" every time. After all, for every task, it is possible to find someone who is really bad at it, and we should not need to keep repeating we are not talking about _that_ person.

[exe34]

ah you're a mathematician I see. without loss of generality, let's assume the bottom half doesn't exist. therefore only the top half needs to be bested and the task is now impossible. qed.