[Tenoke]

> It just so happens that sometimes that non-deterministic text aligns with reality, but you don’t really know when and neither does the model.

This is overly simplistic and demonstratably false - there's plenty of scenarios where a model will sell something false on purpose (e.g. when joking) and will tell you it was false with high probability correctly whether it was false or not after that.

However you want to frame it - there's clearly a more accurate than chance evaluation of truthfulness.

[coffeebeqn]

I don’t see how A follows from B. Being able to lie on purpose doesn’t in my mind mean that it’s also able to tell when a statement is true or false. The first one is just telling a tale which they are good at

[Tenoke]

But it is able to tell if a statement is true or false, as in it can predict whether it is true or false with much above 50% accuracy.

[andrepd]

The model has only a linguistic representation of what is "true" or "false"; you don't. This is a limitation of LLMs, human minds have more to it than NLP

[j_maffe]

LLMs are also more than NLP. They're deep learning models.

[porridgeraisin]

What? Yes the modelling technique falls under "deep learning" but it still very much processes language and language only, making it NLP.

Yes yes, language modelling ends up being surprisingly powerful at scale, but that doesn't make it not language modelling.

[delusional]

A couple of years of this LLM AI hype train has blinded people to what was actually surprising about LLMs. The surprise wasn't that you could make a language model and it wasn't that a language model could generate text. Those are both rather pedestrian observations, and their implementations are trivial. The surprise of LLMs was that contemporary hardware could scale this far and that an un-curated training set turns out to contain a statistically significant amount of truth. Deep learning was interesting because we didn't expect that amount of computation to be feasible at this time in human history, not because nobody had ever thought of it before.

The surprise of the LLM AI was that they were somewhat truthful at all.

[mcv]

The AI revolution has mostly been a hardware revolution. I studied AI in the 1990s, so I knew about neural networks and backpropagation, so when suddenly everybody was talking about "Deep Learning", I wanted to know what was different about it. Turns out: not much. It's mostly just plain old backpropagation on a much larger scale because we have more powerful hardware.

Of course there have still been plenty of meaningful innovations, like the transformer/attention thing, but it's mostly the fact that affordable graphics cars offer massively-parallel floating point calculations which turns out to be exactly what we need to scale this up. That and the sheer amount of data that's become available in the age of the Internet.

[Tenoke]

>The AI revolution has mostly been a hardware revolution.

It's certainly important but this reads as overly simplistic to me. All the hardware we have today won't make an SVM or a random forest scale the way transformers do.