[tgv]

But it also suggests that the neural networks are not the miracle as some proclaim. They get their knowledge from compressing a very large amount of text, but add little in terms of inference. Its success is then a function of its size, but that size cannot grow much. It also ties in with the idea that the current ANNs don't discriminate text, and repeat bad input as readily as anything else.

[naasking]

Compressing nearly all of human knowledge into a desktop PC that can converse in natural language and translate between languages is the miracle.

[tgv]

It's miraculous, but it's not nearly nearly all of human knowledge. Ask anything slightly specialist, and it'll give wrong answers. ChatGPT's translation is also not well developed. Llama doesn't have any, I believe.