[bigyabai]

> We don’t even know how LLMs work

Speak for yourself. LLMs are a feedforward algorithm inferring static weights to create a tokenized response string.

We can compare that pretty trivially to the dynamic relationship of neurons and synapses in the human brain. It's not similar, case closed. That's the extent of serious discussion that can be had comparing LLMs to human thought, with apologies to Chomsky et. al. It's like trying to find the anatomical differences between a medieval scribe and a fax machine.

[ben_w]

> Speak for yourself. LLMs are a feedforward algorithm inferring static weights to create a tokenized response string.

If we're OK with descriptions so lossy that they fit in a sentence, we also understand the human brain:

A electrochemical network with external inputs and some feedback loops, pumping ions around to trigger voltage cascades to create muscle contractions as outputs.

[bigyabai]

Yes. As long as we're confident in our definitions, that makes the questions easy. Is that the same as a feedforward algorithm inferring static weights to create a tokenized response string? Do you necessarily need an electrochemical network with external stimuli and feedback to generate legible text?

No. The answer is already solved; AI is not a brain, we can prove this by characteristically defining them both and using heuristic reasoning.

[ben_w]

> The answer is already solved; AI is not a brain, we can prove this by characteristically defining them both and using heuristic reasoning.

That "can" should be "could", else it presumes too much.

For both human brains and surprisingly small ANNs, far smaller than LLMs, humanity collectively does not yet know the defining characteristics of the aspects we care about.

I mean, humanity don't agree with itself what any of the three initials of AGI mean, there's 40 definitions of the word "consciousness", there are arguments about if there is either exactly one or many independent G-factors in human IQ scores, and also if those scores mean anything beyond correlating with school grades, and human nerodivergence covers various real states of existance that many of us find incomprehensible (sonetimes mutually, see e.g. most discussions where aphantasia comes up).

The main reason I expect little from an AI is that we don't know what we're doing. The main reason I can't just assume the least is because neither did evolution when we popped out.

[ninetyninenine]

George Hinton the person largely responsible about the AI revolution has this to say:

https://www.reddit.com/r/singularity/comments/1lbbg0x/geoffr...

https://youtu.be/qrvK_KuIeJk?t=284

In that video above George Hinton, directly says we don't understand how it works.

So I don't speak just for myself. I speak for the person who ushered in the AI revolution, I speak for Experts in the field who know what they're talking aboutt. I don't speak for people who don't know what they're talking about.

Even though we know it's a feedforward network and we know how the queries are tokenized you cannot tell me what an LLM would say nor tell me why an LLM said something for a given prompt showing that we can't fully control an LLM because we don't fully understand it.

Don't try to just argue with me. Argue with the experts. Argue with the people who know more than you, Hinton.

[staticman2]

"In that video above George Hinton, directly says we don't understand how it works."

That isn't what Hinton said in the first link. He says essentially:

People don't understand A so they think B.

But actually the truth is C.

This folksy turn of phrase is about a group of "people" who are less knowledgeable about the technology and have misconceptions.

Maybe he said something more on point in the second link, but your haphazard use of urls doesn't make me want to read on.

[ninetyninenine]

Take a closer look at BOTH videos. Not just the first one. He literally says the words “don’t” and “understand” in reference to LLMs.

I watch a lot of video interviews on hinton I can assure you that “not understanding” is 100 percent his opinion both from the perspective of the actual events that occurred and as someone who knows his general stance from watching tons of interviews and videos about him.

So let me be frank with you. There are people smarter than you and more eminent than you who think you are utterly and completely wrong. Hinton is one of those people. Hopefully that can kick start the way you think into actually holding a more nuanced world view such that you realize that nobody really understands LLMs.

Half the claims on HN are borderline religious. Made up by people who unconsciously scaffold evidence to support the most convenient view.

If we understood AI completely and utterly we would be able to set those weights in a neural net into values that give us complete and total control over how the neural net behaves. This is literally our objective as human beings who created the neural net. We want to do this and we absolutely know that their exists a configuration of weights in reality that can help us achieve this goal that we want so much.

Why haven’t we just reached this goal? Because we literally don’t understand how to reach this goal even though we know it exists. We. Don’t. Understand. It is literally the only conclusion that follows given our limited ability to control LLMs. Any other conclusion is ludicrous and a sign that your logical thought process is not crystal clear.

[staticman2]

I'm not going to waste my time clicking a second video link but if Gemini can be believed he said "we don't know exactly how they work" which somehow became

"We don’t even know how LLMs work. "

In your retelling, an exaggeration which rightfully led to pushback.

Aside from that I don't know what conversation you think we are having.

[ninetyninenine]

Yeah let’s not talk at all. You’re wasting both your own time and my time by responding to me without clicking on my links and trying to understand what I say.

Just leave. Don’t bother communicating with me again.

[staticman2]

As I said your interpretation of the first link was implausible and I wasn't going to click on a second.

[bigyabai]

Hinton invented the neural network, which is not the same as the transformer architecture used in LLMs. Asking him about LLM architectures is like asking Henry Ford if he can build a car from a bunch of scrap metal; of course he can't. He might understand the engine or the bodywork, but it's not his job to know the whole process. Nor is it Hinton's.

And that's okay - his humility isn't holding anyone back here. I'm not claiming to have memorized every model weight ever published, either. But saying that we don't know how AI works is empirically false; AI genuinely wouldn't exist if we weren't able to interpret and improve upon the transformer architecture. Your statement here is a dangerous extrapolation.

> you cannot tell me what an LLM would say nor tell me why an LLM said something for a given prompt showing that we can't fully control an LLM because we don't fully understand it.

You'd think this, but it's actually wrong. If you remove all of the seeded RNG during inference (meaning; no random seeds, no temps, just weights/tokenizer), you can actually create an equation that deterministically gives you the same string of text every time. It's a lot of math, but it's wholly possible to compute exactly what AI would say ahead of time if you can solve for the non-deterministic seeded entropy, or remove it entirely.

LLM weights and tokenizer are both always idempotent, the inference software often introduces variability for more varied responses. Just so we're on the same page here.

[int_19h]

> If you remove all of the seeded RNG during inference (meaning; no random seeds, no temps, just weights/tokenizer), you can actually create an equation that deterministically gives you the same string of text every time.

That answers the "what", but not the "why" nor the "how exactly", with the latter being crucial to any claim that we understand how these things actually work.

If we actually did understand that, we wouldn't need to throw terabytes of data on them to train them - we'd just derive that very equation directly. Or, at the very least, we would know how to do so in principle. But we don't.

[ninetyninenine]

> But saying that we don't know how AI works is empirically false;

Your statement completely contradicts hintons statement. You didn’t even address his point. Basically you’re saying Hinton is wrong and you know better than him. If so, counter his argument don’t restate your argument in the form of an analogy.

> You'd think this, but it's actually wrong.

No you’re just trying to twist what I’m saying into something that’s wrong. First I never said it’s not deterministic. All computers are deterministic, even RNGs. I’m saying we have no theory about it. A plane for example you can predict its motion via a theory. The theory allows us to understand and control an airplane and predict its motion. We have nothing for an LLM. No theory that helps us predict, no theory that helps us fully control and no theory that helps us understand it beyond the high level abstraction of a best fit curve in multidimensional space. All we have is an algorithm that allows an LLM to self assemble as a side effect from emergent effects.

Rest assured I understand the transformer as much as you do (which is to say humanity has limited understanding of it) you don’t need to assume I’m just going off hintons statements. He and I knows and understands LLMs as much as you even though we didnt invent it. Please address what I said and what he said with a counter argument and not an analogy that just reiterates an identical point.

[int_19h]

The fact that it doesn't operate identically or even similarly on the physical layer doesn't mean that similar processes cannot emerge on higher levels of abstraction.

[hnfong]

Pretty sure in most other contexts you wouldn't agree a medieval scribe knows how a fax machine works.

[ninetyninenine]

Analogies aren’t proof. Like if an analogy doesn’t apply in certain context it is not a reflection of the actual situation. It just means the analogy is bad and irrelevant.

Often people who don’t know how to be logical end up using analogies as proof. And you can simply say that the analogy doesn’t apply and is inaccurate and the whole argument becomes garbage because analogies aren’t logical basis for anything.

Analogies are communication cools to facilitate easier understanding they are not proofs or evidence of anything.