[sfn42]

Being confidently wrong isn't even the problem. It's a symptom of the much deeper problem that these things aren't AI at all, they're just atocomplete bots good enough to kind of seem like AI. There's no actual intelligence. That's the problem.

[indigoabstract]

I think what matters most is that we now know that it's possible, that a computer mimicking most of our abilities (but not all) which we have long considered intelligent is obviously possible in some indeterminate future.

It's not obvious how long until that point or what form it will finally take, but it should be obvious that it's going to happen at some point.

My speculation is that until AI starts having senses like sight, hearing, touch and the ability to learn from experience, it will always be just a tool/help/aider to someone doing a job, but could not possibly replace that person in that job as it lacks the essential feedback mechanisms for successfully doing that job in the first place.

[Workaccount2]

My favorite "paper" on AI pretty accurately describes this line of thinking

https://ai.vixra.org/pdf/2506.0065v1.pdf

[ninetyninenine]

No. The experts in the field are past this argument. People have moved on. It is clear to everyone who builds LLMs that the AI is intelligent. The algorithm was autocomplete, but we are finding as an autocomplete bot is basically autocompleting things with humanity changing intelligent content. Your opinion is a minority now and not shared by people on the forefront of building these things. Your holding onto the initial fever pitched alarmist reaction people had to LLMs when it first came out.

Like you realize humans hallucinate too right? And that there are humans that have a disease that makes them hallucinate constantly.

Hallucinations don’t preclude humans from being “intelligent”. It also doesn’t preclude the LLM from being intelligent.

[dns_snek]

> Your opinion is a minority now and not shared by people on the forefront of building these things.

Minority != wrong, with many historic examples that imploded in spectacular fashion. People at the forefront of building these things aren't immune from grandiose beliefs, many of them are practically predisposed to them. They also have a vested interest in perpetuating the hype to secure their generational wealth.

[ninetyninenine]

It doesn’t but I would argue that evidence is in favor of the majority.

The ai can easily answer correctly complex questions NOT in its data set. If it is generating answers to questions like these out of thin air which fits our colloquial definition of intelligence.

[dns_snek]

LLMs also fail to answer simple questions that require a minimal amount of reasoning which demonstrates that they do not have a working model of the world. Their answers to factual questions will change depending on how you phrase the question, even if the crux of the question is identical:

"Is X true" -> "Yes, X is true."

"Is X a myth?" -> "Yes, X is a myth"

"Is Y a myth?" (where X = Y, rephrased) -> "No, Y is true"

Even when they're provided with all the facts required to reach the correct answer through simple reasoning, they'll often fail to do so.

Worse still, sometimes they can be told what the correct answer is, with a detailed step-by-step explanation, but they'll still refuse to accept it as true, continuing to make arguments which were debunked by the step-by-step explanation.

All state of the art models exhibit this behavior, and this behavior is inconsistent with any definition of intelligence.

[sfn42]

I kind of started this thread but didn't have the energy to argue about it. You provided the exact argument I wanted, thanks for that. This is exactly the reason why I am adamant that LLMs are not intelligence.

They do this really impressive stuff like generate code and hold conversations that makes them seem intelligent, but then they fail at these extremely basic tasks which, to me, proves that it's all an illusion.

It doesn't understand the instructions you give it, it doesn't even understand the answer it gives you. It just consumes and generates tokens. Sure it works pretty well and it's pretty cool stuff, but it's not AI.

[ninetyninenine]

So. Humans hallucinate too and get simple shit wrong all the time too. That’s the current problem with LLMs we know it gets shit wrong. We also know humans get shit wrong and make hallucinatory claims but that doesn’t make us classify humans as not intelligent.

The fact of the matter is that as retarded and as stupid as the LLM is the fact that it’s so prevalent in the world today is because it gets answers right. We ask it things not in its training data and it produces an answer out of a range of possibilities that is to low probability to be produced by ANY other thing other than actual reasoning and logic.

You need to see nuance here and make your assessment of LLMs NOT based on singular aspects of facts. LLMs get shit wrong all the time they also get shit right all the time and so do humans. What does that look like holistically?

Look at the shit it’s getting right . If it’s getting stuff right that’s not in the training data then some mechanism in there is doing actual “thinking” and when it gets shit wrong well, you get shit wrong too. All getting shit wrong does to you is make you a dumbass it doesn’t make you not an intelligent entity. You don’t lose that status as soon as you do something incredibly stupid which I’m sure you’ve done often enough in your life to know the difference.

[dns_snek]

So this has nothing to do with "getting things wrong" and everything to do with why they get things wrong.

They get simple, rephrased, but conceptually equivalent questions really wrong and they do this:

1. while the context already contains their previous answer to the original question (which was correct),

2. while the context contains all background information on the topic that would allow an intelligent being to arrive at the correct answer through simple logical deduction,

3. without recognizing or acknowledging that they provided a conflicting answer (without being prompted),

4. while denying that the two answers are contradictory if that fact is pointed out to them,

5. while fabricating a list of bogus reason justifying a different answer if pressed for an explanation.

That's one common failure mode, the other common failure mode is where they uncritically accept our own erroneous corrections even when the correction contain obviously flawed reasoning.

This behavior demonstrates a fundamental lack of conceptual understanding of the world and points at rote memorization in the general case. Maybe LLMs develop a more conceptual understanding of a certain topic when they've been benchmaxxed on that topic? I don't know, I'm not necessarily arguing against that, not today anyway.

But these errors are a daily occurrence in the general case when it comes to any topic they haven't been benchmaxxed for - they certainly don't have a conceptual understanding of cooking, baking, plumbing, heating, electrical circuits, etc.

[CodexArcanum]

LLMS don't "hallucinate" they generate a stochastic sequence of plausible tokens that, in context when read by a human, are a false statement or nonsensical.

They also dont have an internal world model. Well I don't think so, but the debate is far from settled. "Experts" like the cofounders of various AI companies (whose livelihood depends on selling these things) seem to believe that. Others do not.

https://aiguide.substack.com/p/llms-and-world-models-part-1

https://yosefk.com/blog/llms-arent-world-models.html

[ninetyninenine]

I’m not talking about startups with financial stake. I’m talking about academics and researchers who have zero financial stake and are observing the phenomenon. It is utterly clear now that stochastic parroting is not what’s going on.

[kilpikaarna]

> It is clear to everyone who builds LLMs that the AI is intelligent.

So presumably we have a solid, generally-agreed-upon definition on intelligence now?

> autocompleting things with humanity changing intelligent content.

What does this even mean?

[ninetyninenine]

We do it’s fuzzy but we do. You point to a rock all humans say it’s not intelligent. You point to a human all humans say it is intelligent.

Because we can do this, by logic a universally agreed upon definition exists. Otherwise we wouldn’t be able to do this.

Of course the boundaries between what’s not intelligent and what is, is where things are not as universally agreed upon. Which is what you’re referring to and unlike you I am charitably addressing that nuance rather then saying some surface level bs.

The thing is the people who say the LLM (which obviously exists at this fuzzy categorical boundary) is not intelligent will have logical paradoxes and inconsistencies when they examine there own logic.

The whole thing is actually a vocabulary problem as this boundary line is an arbitrary definition given to a made up word that humans created. But one can still say an LLM is well placed in the category of intelligent not by some majority vote but because that placement is the only one that maintains logical consistency with OTHER entities or things all humans place in the intelligent bucket.

For example a lot of people in this thread say intelligence requires actual real time learning, therefore an LLM is NOT intelligent. But then there are humans who literally have anterograde amnesia and they literally cannnot learn. Are they not intelligent? Things like this are inconsistent and it happens frequently when you place LLMs in the not intelligent bucket.

State your reasoning for why your stance is "not intelligent" and I can point out where the inconsistencies lie.

[wila]

that you're arguing with an LLM :)

[ninetyninenine]

he's not.

[NoGravitas]

IOW, the realistic position is not held by the majority of people whose paychecks depend on it being wrong. I'm shocked.

[mrguyorama]

Also people tend to use "Expert" wrong in the AI world. No, your programmer who has ten years experience integrating and deploying ML models is not an "AI Expert", they are programmers with expertise with some libraries. Building a system that integrates with ffmpeg for media conversion does not make you a "Video compression expert".

Go check out anthropic's careers page and see just how few positions even require a formal training in statistics.

Meanwhile I don't see a lot of real statisticians who are that hyped about LLMs. More importantly, it feels like there aren't even that many scientists at the AI companies.

Your average programmer does not have nearly the "question your assumptions and test your beliefs" training that an actual scientist has, which is funny since nearly every bug in code is caused by an assumption you shouldn't have made and should have tested.

[ninetyninenine]

When I say experts. I refer to academia.

You're shocked because you hallucinated an assumption of something I never claimed.

Hallucinations? Does that sound similar to something?

[mcv]

Yeah, but how much of that is wishful thinking? If your job depends on believing this is real intelligence, you're more likely to believe that.

[eCa]

> Like you realize humans hallucinate too right?

A developer that hallucinates at work to the extent that LLMs does would probably have issues getting their PRs past code reviews a lot.

[ninetyninenine]

A person who has schizophrenia and hallucinates to a greater extent than LLMs are clearly defective and not intelligent or sentient.

Because of this we should euthanize all schizophrenics. Just stab them to death or put a bullet in their heads right? I mean they aren’t intelligent or sentient so you shouldn’t feel anything when you do this.

I’m baffled as to why people think of this in terms of PRs. Like the LLM is intelligent but everyone’s like oh it’s not following my command perfectly therefore it’s not intelligent.

[bigstrat2003]

They would have issues even remaining employed. AI defenders are very quick to point out "humans mistakes too", but that is a false equivalence because humans learn. If a junior makes a really stupid mistake, when I show him the correct way he won't make that mistake again. An AI will, because (as people correctly point out) it has no actual intelligence.

[ninetyninenine]

There’s examples of humans who can’t learn. Have you seen the movie memento.

There are cases where humans lose all ability to form long term memories and outside of a timed context window they remember nothing. That context window is minutes at best.

According to your logic these people have no actual intelligence or sentience. Therefore they should be euthanized. You personally can grab a gun and execute each of these people one by one with a bullet straight to the head because clearly these people have no actual intelligence or sentience. That’s the implication of your logic.

https://en.m.wikipedia.org/wiki/Anterograde_amnesia

It’s called anterograde amnesia. Do you see how your logic can justify gassing all these people holocaust style?

When I point out the flaw in your logic do you use the new facts to form a new conclusion? Or do you rearrange the facts to maintain support for your existing conclusion?

If you did the later I hate to tell you this, it wasn’t very intelligent. It was biased. But given that you’re human, that’s what you most likely did and it’s normal. But pause for a second and try to do the former of using the new facts to form a different more nuanced conclusion.