|
|
|
|
|
|
by krapp
1098 days ago
|
|
|
Most people have never actually dealt with something like modern LLMs, so we haven't really developed the proper language to describe them and how they behave. It's either too simplistic and reductive (stochastic parrot, xerox machine) or presupposes sentience and intent ("fabricates","hallucinates", etc.) |
|
|
I think we are focusing on the model too much and miss the real hero - language. The corpus of text these models are trained on is a marvel of human creativity. This cultural artefact is the diff between primitive and modern humans. And it is the diff between a random initialisation and a trained GPT-4. Maybe the brain or the model don't matter, but what you train them on.
Even more, language is special. Ideas are self replicators, they have a lifecycle, they have evolutionary pressure to improve. Ideas travel a lot. No single human can recreate this knowledge, it is the result of massive search. I'd say more than 99% of human intelligence is based on applying ideas invented by someone else. So let's be more lenient on the parroting accusations. AIs can be smart if they get feedback, like AlphaZero, but without feedback they of course have to parrot.