[Leonard_of_Q]

I've read a bit through your comment history which gives me the impression of a mostly rational individual with whom I agree on some things while disagreeing on others. You don't seem to be a raving anti-LLM crusader nor come across as a starry-eyed LLM fanboi. I therefore conclude that these last rants were, to speak with Ebenezer, the result of an undigested bit of beef, a blot of mustard, a crumb of cheese, a fragment of an underdone potato rather than a farewell.

Assuming this to be the case my real question is: what makes you so sure these things don't "think"? This question can only be answered if we first know what "thinking" actually entails. Sure, LLMs are mechanistic and deterministic, feed them the same quote and seed and they'll produce the same output, token for token. If what they do is "thinking" - albeit mechanistically - then it seems to give lie to the concept of free will since the output for a given input only depends on the seed value. Surely humans don't 'think' like that? Well... who knows? The 'neural network' in human brains is far more complex than the ones used to run LLMs while LLMs can have access to more 'factoids' than the average human. What comprises 'thinking' as we do it? What would happen if you give, say, the neural circuitry in a rat brain access to enough storage to contain the training data used in current LLMs? Can a machine ever be made to 'think' or is that something which will always be limited to living organisms? If the answer is 'yes' we're back at the definitional question of what 'thinking' entails, if it is 'no' we're entering more in the realm of metaphysics and religion.

I don't know what 'thinking' entails, I just know I do it. I therefore can not definitely state whether LLMs 'think' or 'reason' but I can apply reason to what I observe and know about how these things work. Those observations and that knowledge lead me to conclude that, absent some metaphysical or religious veto these models can be made to 'think' and might already be doing so.

[scrollaway]

You phrased it better (and with a lot more patience) than I ever could muster.

So many people I meet are so deeply convinced LLMs absolutely cannot physically think, because they define "thinking" as "that thing you do with your human brain where neurons are involved", and they define "LLM thinking" as "that thing ChatGPT does where it says it's thinking but it's actually just detached inference".

The underlying assumption is usually two-fold:

1. That simulated thinking is not thinking.

2. That "LLM thinking" is always only defined as Chain of Thought.

Well, 1 is a pretty useless stance to have, because it removes space for any useful definition of what thinking is. And 2 is simply false, as presented by Anthropic here: https://www.anthropic.com/research/global-workspace

[blooalien]

To be 100% honest, I really don't want to have this (or any other) conversation with you (or anyone else) right now (or possibly ever; I'm just sick and tired of people and their hateful bullshit), but purely out of respect for your fair and balanced reply I'll try to respond here in a civil and honest way to the best of my ability. This really is the last response I intend to make to anything anyone says on the topic, so I'll try to be thorough. If you want to respond to anything I say here, feel free, but don't expect a reply after this one (I may possibly, but probably not) . That having been said, here goes...

> ... "impression of a mostly rational individual with whom I agree on some things while disagreeing on others."

I try, really I do. It's gotten really hard these days. You're welcome to agree or disagree; Totally normal and expected. I just get tired of getting shut-down on every little thing I say by so many people who have sometimes less than zero experience in the topic they claim absolute certainty about, no matter if I can trot out a parade of facts proving my points. This inevitably leads to stress that is no longer as easy to just "brush off" as it used to be. Sorry for that.

> "You don't seem to be a raving anti-LLM crusader nor come across as a starry-eyed LLM fanboi."

You're right. I'm neither. I am actually quite impressed and amazed with what LLMs are capable of (especially in the hands of skilled and knowledgable users) but I also understand fully that there are tradeoffs involved and responsibilities involved in the usage of such tools. I do believe they (and other "AI" related technologies) have huge potential for both good and bad (largely dependent upon the user and their intent) and like any new tool, I genuinely do hope this one finds more of the good use than the bad, but more and more I'm feeling like it's just gonna get weaponized against society at large. Sad, but nothing I can say or do will change it. I'm fully convinced of that at this point.

> "what makes you so sure these things don't "think"?" ... <more stuff said here> ... "If the answer is 'yes' we're back at the definitional question of what 'thinking' entails, if it is 'no' we're entering more in the realm of metaphysics and religion."

So, in my mind, "thinking" is a much more "active" process than the "calculation" done by a machine just mechanistically working through a bunch of math. Does a desktop calculator "think"? Does a mechanical device like an Abacus or anything else that can "do math" without electronics? Calculation isn't necessarily "thinking", even though thinking can (and often does) result in calculation.

Now, where I'm coming from with my assertion that LLMs don't actually think is due to a few factors. First off, I've been learning the mathematics involved in how these things work for a very long time (decades now actually; as "neural network" technology and ideas is truly not a new thing), and while it's really amazing stuff, it's not magic. It's just math. Really fancy and complex math, but still just math. As soon as the math stops being done, the "thinking" stops. Does a brain ever stop thinking? I get the impression that until death it's kinda always active, even when you sleep. Not so with an LLM. You give it input, a buncha fancy math gets done by a really powerful "calculator" (computer), it responds with output, then it stops until it gets another "trigger" to start calculating some more.

There's some very real flaws in seeing that process as thinking however, even if you're only talking about that time during which the calculations are taking place. The problem I see there is that the LLM cannot "second guess" itself or worry about whether it might be incorrect about something. It just forges ahead with the calculations and gives the end result to the user, right or wrong, as it was designed to do. It has no "skin in the game" or reason to care (even if it had the ability to care) and it's got no real sense of "self" or the world or anything. It's just doing some really amazing math that results in an illusion of a thought process.

That having been said, I'm firmly convinced that even as these things stand now, they can absolutely assist humans in their thought processes if used properly and judiciously with full understanding of their limitations and weaknesses taken into account. I just don't believe that "more of the same" will somehow magically become "sentient" someday without a huge advance in both the hardware and software technologies it's built upon (on the level of the "positronic brain" or some kinda hand-wavy "quantum technology" science fiction concept). Pretty darn certain that more massive "AI data centers" aren't gonna lead to a "magical thinking machine" with the current forms of "AI" we're working with.

> "Those observations and that knowledge lead me to conclude that, absent some metaphysical or religious veto these models can be made to 'think' and might already be doing so."

Now, this here I can actually agree with, other than the "might already be doing so" part. They're not (yet). I'm really quite sure of that, knowing what I know about how these things work. They really are fantastic at faking it these days though, as evidenced by how many people truly are buying into the AI company CEO hype about AGI/ASI. I think that LLMs can absolutely be one part of a machine that's capable of a simulation of "thought" that could really be good enough to qualify as some form of "the real thing" on some level, and that may even someday (soon even?) surpass the capabilities of humans in that regard. It'll require some different ways of doing things though, and some combinations of classic traditional computing with a wide range of related "AI" technologies including LLMs, neural nets, vision models, etc, etc, and it'll have to be put in some sort of active state of operation where it's capable of doing the "thinking" and "learning" process continuously the way an actual brain does. I think it'll also help to give it access to a continuous input stream similar to how a brain has access to near constant input as well.

Anyone that wants to really know how this stuff works "under the hood" is welcome to ask an LLM about it. Many of 'em are actually quite good at explaining themselves, starting from "first principles" if you ask 'em to "keep it simple" all the way down through the deep mathematics involved. I encourage folks to have that discussion with several of their favorite LLMs if for no other reason than more knowledge about the topic is a good thing. Just be aware that they can at times say things that are actively incorrect and they will often say such things with great certainty (and sometimes even try to argue with you about it if you call them out on it). Always check your own (and the LLM's) knowledge against known verifiable provable facts. This stuff is all heavily documented and readily available "out there" on the Web with not too terribly much heavy searching required.

To summarize; I don't think it's impossible to create a "thinking machine" using these technologies. I just don't believe we're even remotely nearly as close to it as the AI mega-corporations would have us all believe. I might be wrong about everything I've said here, or I could be 100% correct. Dunno; No longer care either way really. I've said my piece and I'm done now. Bring on our AI overlords, for better or worse. I can't stop it either way.

[scrollaway]

I don't know why you're saying you're "tired of people's hateful bullshit". I originally responded that your point was off, and you're the one writing these comments: https://news.ycombinator.com/item?id=48886350

Take a breather, my guy. Stop saying in every comment that "you're done" and "this is your last response" and actually go touch a bit of grass. You do not owe anyone on this website a response, and the only thing your comments are for (when they're good and not hostile) is contributing to a discussion. Nobody here cares whether you respond or not.

Anyway, not to detract - this last comment was good. I just think maybe you're putting intent in people's mouths where there is none. For example, I certainly don't believe the bullshit Altman and Amodei are putting out. I just disagree the LLMs don't think.

And yes I also understand the math behind them. But it being math doesn't mean there cannot be emergent behaviour, just like there is emergent behaviour after the layers upon layers of biology in humans, resulting in thinking. "It's not magic, it's just [biology]" applies to us as well.

"But brains are the most complex machines in the universe" some might say -> Right, but who's to say the thinking we do requires machines as complex as our brains are? Our brains move muscles, help us breathe and manage millions of other invisible innate processes, many things LLMs do not need to do. And LLMs have a very, very deep access to and understanding of language, which is argued to be a significant contributor to how humans think (based on studies on nonverbal humans).

> As soon as the math stops being done, the "thinking" stops. Does a brain ever stop thinking?

Right, so I understand your point in this and there's something to that, but it enters in how thinking is defined in both ways.

First, the calculation not running is at best equivalent to time stopping. It's not like the LLM actually sits there waiting for input. As you said: we continuously receive input, and we do so because our biology is built this way and we are damn energy efficient, so we can afford to continuously and asynchronously process that input. But this is not really related to the process of thinking itself.

So it comes down to what happens during the calculation. And Anthropic's published research on the J-space is pretty damning evidence, IMO, that thinking does happen during the calculation.

[blooalien]

Yeah, you can just fuck right off, too. My reply wasn't for you. It was for the person who wasn't actively trying to rub salt in the wounds and be a worthless little shit.

> I don't know why you're saying you're "tired of people's hateful bullshit". I originally responded that your point was off, and you're the one writing these comments: https://news.ycombinator.com/item?id=48886350

You really can't figure it out? Did I not make myself pretty clear in the comment I was responding to there that it's decades of shitty people like him (and apparently you) that have pushed me to this point? Funny how you point to my comment without noticing or acknowledging at all the fact that it was a response to another hostile comment that was all about just tryin' to be hateful and ignorant and push me even further into hatred.

> Take a breather, my guy.

I'm not "your guy". I actively hate you.

> ... and actually go touch a bit of grass.

And this sorta shit right here is why I hate you and everyone like you. You know what I did yesterday? I turned my computer off and went and "touched grass" (hung out in the park away from technology and people) and just chilled out. You make it sound like I don't love nature or some stupid shit. You don't know me, so just keep your unhelpful advice to yourself.

> I just think maybe you're putting intent in people's mouths where there is none.

So it wasn't your intent to make snide little insinuations and remarks in an attempt to purposely try to rub me the wrong way so as to piss me off even more after I'd finally managed to calm myself down? Yeah, sure. I believe that... It wasn't me who put those remarks in your mouth. You chose to try to rub salt in the wound. Congratulations. You're human garbage. Good for you.

[scrollaway]

Talk to a therapist.

[blooalien]

> Talk to a therapist.

Why? It might help me but it won't fix you and you're the problem here. Your whole intent is to aggravate me into hating you even more than I already do. Well, guess what. You're wasting your time at this point. I couldn't possibly hate you more than literally wishing you didn't exist at all. You've already succeeded at your goal, so you can just shut the fuck up now and go away. Congratulations. You win. Be happy and be gone.