[tinsmith]

I mean, humans do that. We are remarkably contradictory when expressing ourselves, generally speaking, often without realizing it because we'll change our thinking in the moment to fit the current narrative or circumstance. LLMs just put that on blast.

[quickthrower2]

The reason humans do this and LLMs is very different. Humans do it as a social skill / tribe fitting behaviour. Agreeableness. (watch out for that!)

[naasking]

> The reason humans do this and LLMs is very different. Humans do it as a social skill / tribe fitting behaviour.

Tribe fitting doesn't sound as far off from minimizing loss functions as you imply.

[DonaldPShimoda]

I want you to find me a human who (a) has defined a loss function for social interaction and (b) consciously performs the statistical analysis involved in fitting that loss function in social settings.

LLMs do not have cognitive processes. They do not think. They do not choose to obey the requirements of a loss function; it is simply how they work, like any machine. Humans do not work this way, and the difference is fundamental.

[naasking]

> I want you to find me a human who (a) has defined a loss function for social interaction and (b) consciously performs the statistical analysis involved in fitting that loss function in social settings.

Why would they have to do that consciously? Do you think LLMs do this "consciously", if that term even applies? I wouldn't think so. The loss function applies during training, that ultimately defines the weights which guide their "thinking process".

Analogously, human experiences in childhood shape our ultimate neural weights which guide our thinking processes in social situations in adulthood.

> LLMs do not have cognitive processes. They do not think.

You don't know what thinking is mechanistically, so you can't make this claim. I don't know why people keep pretending we have knowledge that we do not in fact have.

[DonaldPShimoda]

> Why would they have to do that consciously? Do you think LLMs do this "consciously", if that term even applies?

Because the only thing an LLM does is apply statistics to inputs to generate outputs. That's literally it. It's just a pile of statistics.

To draw an analogy to humans requires that humans are similarly statistical. LLMs do not have cognitive processes, while humans do, so the analogy obviously requires making some kind of leap between the two if it is to have any merit whatsoever. My request lines up with this: if LLMs are "not so different" from humans, and if LLMs work only based on statistics, then this requires humans also work only based on statistics. I want to see evidence of this.

> You don't know what thinking is mechanistically, so you can't make this claim. I don't know why people keep pretending we have knowledge that we do not in fact have.

Are you suggesting the code behind the LLMs is just, like, ineffable or something? And therefore we can't know how they work, so we get to just make up wild claims about their capabilities, and then smarmily position ourselves as some kind of authority on the matter when other people talk about how they work?

No, I don't think so. We do know how LLMs work, and the way they work is not thinking. They have no agency. Claims to the contrary are, frankly, absolutely absurd. They're just statistics. There's nothing magical about them. You should stop pretending there is.

[naasking]

> To draw an analogy to humans requires that humans are similarly statistical.

Prove they are not.

> Are you suggesting the code behind the LLMs is just, like, ineffable or something?

No, the argument is quite simple. We understand the mathematics of transformers and LLMs, therefore they seem obvious and not at all magical.

By contrast, we do not understand the mathematics behind human cognition, therefore it seems complex and mysterious, and we have only non-rigourous folk concepts like "thoughts" and "feelings" to describe mental phenomena. Therefore you cannot intuitively fathom how to bridge the gap between mathematics and your folk concepts, and so any comparison seems absurd, but note that the absurdity is purely a product of our ignorance of the mathematics of mind. This is a classic god of the gaps fallacy.

Here's how you can logically bridge that gap from a different direction: per the Bekenstein Bound, any finite volume contains finite information; a human is a finite volume, therefore it contains finite information; any finite system can be described by a finite state machine; therefore a human can be described by a finite state machine, which is a mathematical model.

Therefore whatever a "thought" or "feeling" is, will correspond to some mathematical object. Now exactly what kinds of mathematical objects they are is unknown.

However, transformers learn to reproduce a function by learning how to map inputs to outputs. The function mapping inputs to outputs is the human brain's function for producing intelligible human text. Therefore, LLMs are at least partially learning the human brain's function for producing intelligible text.

Whether this requires "thoughts" and therefore LLMs fully or partly reproduces what we refer to as "thoughts" is not yet clear, but what is clear is that we have no basis to claim they have no thoughts, because we don't really know what thoughts are.

[bheadmaster]

> Are you suggesting the code behind the LLMs is just, like, ineffable or something?

No, he's suggesting that the code behind the human brain is "ineffable or something". For all we know, human brains might be "just statistics", "nothing magical about them".

You have no way of knowing that LLMs don't think, because we literally don't know what thinking is.

[MVissers]

Humans do this unconsciously as well. You have some extreme medical conditions like confabulation where people just make up the strangest things.

Split-brain personality people can make up stories when you prompted the other brain, and the current half has to explain why it did something.

I'm baffled also that reverse psychology even works on LLMs, to bypass some of its safeties. I mean.. We're using psychological tricks that work on toddlers and also work on these models.

I'm an amateur neuroscientist as you can see, but find LLMs fascinating.

[vidarh]

Is it? To my knowledge we don't have reliable data on why humans do this. To me it appears as if we spend a significant amount of our time retroactively making up justifications for things even to ourselves for things there's little reason to think we've done based on a conscious decision making at all.

[civilized]

Humans can be consistent if we try. LLMs can't, even when prompted to be consistent, because they don't really understand what it means to be consistent.

[vidarh]

Some humans can be consistent if we try some of the time.

Having done phone support in early parts of my career, I'd strongly dispute any notion that most humans can be consistent if we try for anything more than the shortest periods and following the very simplest of instructions.

Most people are really awful at maintaining the level of focus needed to be consistent, and it's one of the reasons we spend so much time drilling people on specific behaviours until they're it's near automatic instead of e.g. teaching people the rules of arithmetic, or driving, or any other skills and expecting people to be able to consistently follow the rules they've learnt. And most of us still keep making mistakes while doing things we've practised over and over and over.

LLMs are still bad at being consistent, sure, but I've seen nothing to suggest that is anything inherent.

I think one of the biggest issues with LLMs if anything is that they've gotten too good at expressing themselves well, so we overestimate the reasoning levels we should expect from them in other areas. E.g. we're not used to an eloquent answer from someone unable to maintain coherent focus and step by step reasoning because human children don't learn to speak like this before we're also able to reason fairly well, and that makes it confusing to deal with LLMs where relative stage of development of different skills does not match what we expect.

[awestroke]

> Humans can be consistent if we try.

That's a bold statement. Do you have evidence for this?

[nickthegreek]

driving in traffic only works because many other people's actions can be consistently understood.

[awestroke]

But accidents happen in traffic all the time

[daveguy]

This is not as strong of an argument as you seem to think.

According to NHTSA [0] there are about 2.1 accidents per million miles driven. This includes fatality, injury-only and property-damage-only accidents. That is the equivalent of over 99.999% of miles driven without an accident. Over 5 nines of reliably consistent behavior.

[0] https://cdan.nhtsa.gov/tsftables/National%20Statistics.pdf

[alwaysbeconsing]

This is a fair point but if what we're looking for is consistent behavior I think we'd have to consider events that don't result in damage or even rise to being accidents (but that could have, if fortune had frowned) like being cut off in a merge or someone running a no-turn-on-red. Which is of course difficult to really measure.

[Symmetry]

Better to say, I think, that humans are much better at improving their approximation of consistency with mental effort both because we can think silently instead of "out loud, step by step" and because some of the patterns of careful thought we engage in don't get written down naturally as text and so are unlikely to be hit upon by GPTs. The advantages of not being a strict feed foreword network.

That being said, GPT4 can just open its virtual mouth spew forth without reflection and still produce consistent text is clearly superhuman.

[hanselot]

I think framing it this way could actually help us reach the next step with AI, if we ask how we could imbue it with those properties.

A human neural net is constantly bombarded with inputs from many different senses, which firstly, gets prioritized based on prior usefulness. That usefulness is updated all the time, constantly, and that's what any current AI implementation lacks.

1 - The continuous integration of data from all "senses". This one should be self-evident, as obviously, all our senses are constantly barraging our brain with data and it learns to handle that over time, in whichever way your genetic makeup + learned internal cognitive processes dictate it be handled.

2 - The network that decides which data requires which amount of attention, and whether to store it in short or long term memory. This is obviously tied in quite closely to 1, as you need massive amounts of data to understand underlying patterns, and which data is just spam, versus what's really valuable.

3 - And with these two things together come the emergence of improving of approximation of consistency. Which means, this itself is a metric which the agent running the other agents needs to be aware of. Its silly to think the human brain is a single agent. It makes way more sense to see it as various interacting agents that equate to a greater sum than its parts.

Now, that being said, I'm not an expert on AI or Data Science, but this is more or less how my understanding of computational theory of mind meets with biological computing and neural networks. My theory is that, the first actually intelligent AI will be one that is composed of a network that makes decisions on how to spend a unit of iteration. One iteration becomes "an instant" to the AI. Aka, the AI decides to spend one iteration thinking, or spawns a sub-agent (which it is aware will consume resources that other processes might also need access to, but it needs to be able to decide which action to pursue).

So in all honesty, its amazing to me that LLM's on their own have been able to achieve this level of "personhood" despite their being only a tiny subset of the whole that makes up a "conscious" entity.

Edit: Misunderstood parent's point.

[Symmetry]

Right. I sort of worry that because LLMs are able to be so coherent without anything resembling a human's working memory at all, then adding either a crude working memory still working with LLM tokens or adding a more sophisticated one assembled from "chunks"[1] will let an AI based on them ascend to clearly superhuman reasoning with just an architectural improvement and no need for any more flops invested than we already put into GPT4.

[1]https://en.wikipedia.org/wiki/Chunking_(psychology)

[wnkrshm]

Mathematics exists, we can build consistent (up to a point of course, not everything can be free of contradiction) models that are pretty rigid.

[awestroke]

Can you?

[wnkrshm]

I'm not a mathematician, so I'm not at the horizon of mathematics - but I can branch off from what's already there to make what I need (and isn't just searchable) consistent with the rest of it.

You're arguing that recent AI developments are a big deal and I'm not arguing against that. But what anyone stating that needs to answer is why we should think that big deal is a good thing - since humans are using the technology and will control whom it benefits. We don't have a good track record there, historically, which is another are in which we are, sadly, consistent.

[civilized]

Don't large sectors of the economy rely on precisely consistent behavior to operate?

[_heimdall]

Economics depends on looking at large enough datasets that inconsistencies can be averaged it and glossed over. People generally following a pattern and precise consistency are very different

[whyenot]

I don’t think another bold statement, phrased as a question, is what most people would consider “evidence.”

[civilized]

I'm phrasing it as a question because I'm asking you to do a small amount of mental legwork to observe the society around you.

How does the financial industry work? How are people comfortable executing transactions?

Do you generally rely on your bank account to not randomly fluctuate in its balance without cause?

Do you work in the tech industry? Do you rely on computers and algorithms and software to do things humans promised they would do?

All of this requires a very high level of consistency either in humans or in tools they have created.

[vidarh]

To me, none of what you mention requires a whole lot of consistency other than in as much as we understand that we are horribly inconsistent we have a whole lot of ceremony and processes built up around what we do to mitigate the wild inconsistencies in the quality of work we do.

[_heimdall]

It's extremely interesting to me that all the examples here only point to humans being consistent in faith of the systems and tools we build

[doctor_eval]

> How does the financial industry work? How are people comfortable executing transactions?

Because there are serious checks and balances? Because of double entry bookkeeping, reconciliation, audits and, ultimately, prisons? Systems of governance put into place with the explicit goal of ameliorating the vagueness of individual humans?

[mistermann]

Very high = ?%?

[bmacho]

No, you can't, in any meaningful sense. Even the biggest bigots, with the most stable beliefs, like RMS or the Pope have a lot of contradictory beliefs. (At least, I think so. I don't have any evidence for this.)

Also any strategy that you might came up and is simple enough for you to follow is trivially followable for chatGPT as well.

[akiselev]

Logical consistency is not the same thing as probabilistic consistency.

[smusamashah]

Humans are not mere LLMs. But it's nice to think we are. May be we can program other humans with carefully constructed prompts alone. May be hypnosis is just a prompt injection attack.

[chongli]

May be we can program other humans with carefully constructed prompts alone

Isn’t that what scammers, catfishers, con artists, and marketers do?

[bmelton]

I mean, it's what parents TRY to do

[coldtea]

It's also what makes a human. Without what parents "try to do" (teach the kids to human), the little people would just be like ferral animals, without language, human context, sense of society, and so on.

[_heimdall]

That's raises the interesting question of whether everything that makes us human is learned behavior.

If it all really is nurture rather than nature we're much more fragile than people may realize and only ever one generation away from going ferral

[seszett]

> only ever one generation away from going ferral

That doesn't seem right to me, as we cannot survive childhood without adults. Young humans are ready for life in society long before they are able to survive on their own without support from the society around them.

We have been depending on the continuation of some form of society for likely hundreds of thousands of years, not just for staying civilised but simply for survival.

[Viliam1234]

Nurture vs nature is a wrong way to look at things. Without the human nature, the nurture would not work. Try to raise a member of a different species as a human...

[doctor_eval]

Not to mention cult leaders.

[ChatGTP]

No

[PartiallyTyped]

> May be we can program other humans with carefully constructed prompts alone.

Is that not what reasoning is in a debate?

> May be hypnosis is just a prompt injection attack.

No need to go into hypnosis. Mere prompts can inject false memories. This has been proven multiple times [1].

[1] The Brain: The Story of You, David Eagleman.

[beardedetim]

The pseudo-science of NLP (_neuro-lingustic programming_) or "conversational hypnosis" is based on the believe that you _can_ program people with carefully constructed prompts alone. Not saying I _believe_ NLP to be valid (_hence the pseudo-science callout_) but I did fall down the Milton H Erickson wormhole 10-20yrs ago so can't say I never fell for it myself.

[quickthrower2]

The real NLP is getting people to believe in NLP

[dsr_]

... and that you can do this over the course of a few minutes or hours, without the subject noticing or objecting.

[TuringTest]

So, what was previously called 'persuading'.

[quickthrower2]

I do feel like a stage hypnotist when prompt engineering an LLM!

[eurekin]

... or a cable TV

[bheadmaster]

> We are remarkably contradictory when expressing ourselves, generally speaking, often without realizing it because we'll change our thinking in the moment to fit the current narrative or circumstance. LLMs just put that on blast.

Exactly.

It seems to me that today's LLMs are on the level of human children. Children often say random bullshit. They try to figure out things, and succeed to some level, but fail spectacularly above that, regressing to whatever connection they can sense. It's only that human children most often learn to fear mistakes, so they stop expressing themselves so freely.

LLMs seem like human children, but without the fearing mistakes part - they just spew whatever comes to their mind, without filter (except those """ethical""" filters built in by OpenAI).

[pydry]

There's a retrospective mechanism working within human intelligence that is very evidently not in play with LLMs.

Humans are capable of creating cargo cults but it seems LLMs are destined for it.