[nuancebydefault]

I believe most confident statements people make, are established the same way. There are some anchor points (inputs and vivid memories) and some part of the brain in some stochastic way dreams up connections. Then we convince ourselves that the connections are correct, just because they match some earlier seen pattern or way of reasoning.

[zdragnar]

The basis of human irrationality is not tied to the basis of LLM irrationality.

LLMs don't get to make value judgements, because they don't "understand". They predict the subsequent points of a pattern given a starting point.

Humans do that, but they also jade their perception with emotive ethics, desires, optimism and pessimism.

It is impossible to say that two humans with the exact same experience would always come to the same conclusion, because two humans will never have the exact same experience. Inputs include emotional state triggered by hormones, physical or mental stress, and so forth, which are often not immediately relevant to any particular decision, but carried over from prior states and biological processes.

[svachalek]

Just because humans have additional sources of irrationality doesn't mean they don't also have irrationality based on the same lack of self-awareness that LLMs exhibit.

[nuancebydefault]

I could understand that argument as follows: LLMs fill in the gaps in a creative but predictable way. Humans fill in the gaps in creative but unpredictable ways. The creativeness level is affected by the ad hoc state of the brain.

I understand that you relate judgement, ethics and emotions to 'understanding'. I'm not convinced. Emotions might as well be an effect of pattern matching. You hear a (pattern matched) type of song, you feel a certain way.

[lo_zamoyski]

Conversely, human beings with varying particular experiences can come to the same conclusions, because human cognition can abstract from particulars, while LLMs are, at best, statistical and imagist. No two of us ever experience the same set of triangles, but abstraction allows us to form concepts like "triangularity", which means we can understand what it means to be a triangle per se (something that is not concrete or particular, and therefore cannot be visualized), while an LLM can only proceed based on the concrete and particular data of input triangles and derivations introduced into the model. It can never go "beyond" the surface features of the training model's images, as it were, and where the appearance of having done so occurs, it is not via abstraction, but by way of product of human abstraction. From the LLM's perspective, there is no triangle, only co-occurrence of features, while abstraction goes beyond features, stripping them away to obtain the bare, unimaginable form.

[the_gipsy]

Different LLMs can also come to the same conclusions, that is, predict the same strings of tokens (in the meaning).

Sure, they're a far way from the capacity of humans in terms of short term training. But there is literally nothing that indicates they can't "think" (understand, reason, abstract, whatever word you wanna put in italics) because nobody can explain what it really means, because: it's all just predicting. Text happens to be super useful for us to evaluate certain aspects of predicting.