[RjQoLCOSwiIKfpm]

Anecdotally, I can say that their effect is already bullshitting indeed:

When being in a random conversation on the "normal kind-of-average people Internet" (Twitch chat), discussing a subject where the host doesn't know a certain word, people already are pasting ChatGPT definitions of the word into the chat, believing it's some kind of dictionary or whatever.

If you tell them it could be completely making things up they'll be like "yeah dunno but it can be a nice overview of the topic".

So people will treat it like the new Google, with no idea (or no concern?) that all it does is mash words together because they're likely to occur next to each other in an arbitrarily defined reference dataset of text which was likely downloaded from random places all over the Internet.

[jodrellblank]

> "with no idea (or no concern?) that all it does is mash words together because they're likely to occur next to each other in an arbitrarily defined reference dataset of text"

You propagate this meme with no idea (or no concern?) that it's incorrect. LLMs are not Markov chains.

[RjQoLCOSwiIKfpm]

Feel free to prove me wrong with sources :)

[jodrellblank]

https://youtu.be/qbIk7-JPB2c about 5 minutes in.

> People say it doesn't have a world model but it's not as clean cut as that, it absolutely could build an internal representation of the world and act on it as it progresses through the sentence temporally. Beware of trillion-dimensional space and its surprises, it's very hard for humans to reason about. [...] We shouldn't think about those neural networks as learning simple concepts like 'Paris is the capital of France'; it's doing much more like operators, it's learning algorithms. Inside it, it's not just retrieving information, not at all, it's built internal representation that allows it to reproduce the data that it has seen succinctly. Really you shouldn't think about it as pattern matching and just trying to predict the next word, yes it was trained to predict the next word but what emerged out of this is a lot more than just a statistical pattern matching object. We need to think about it as learning algorithms. [..] it's something very different from what we are used to.

- Sebastien Bubeck, Sr. Principal Research Manager in the Machine Learning Foundations group at Microsoft Research

[RjQoLCOSwiIKfpm]

Thanks, will have a look! :)

[actionfromafar]

No concern for sure.