[dekhn]

The real question to me is: in the next decade, as ML researchers roll out progressively more sophisticated systems, we can expect that generative systems- which may actually be "only stochastic parrots"- are going to create works that would fool any reasonable human being.

At what point does a stochastic parrot fake it till it makes it? Does it even matter? We can imagine that, within 10 years, we'll have a fully synthetic virtual human simulator- a generative AI combined with knowledge base, language parsing, audio and video recognition, basically a talking head that could join your next technical meeting and look like full contributor. If that happens, will the Timnits and the Benders of the world admit that, perhaps, systems which are indistinguishable from a human may not just be parrots, or perhaps, we are just sufficiently advanced parrotS?

Seen from that perspective, the promoters of stochastic parrots would seem to be luddites and close-minded, as well as discouraging legitimate, important, and valuable scientific research.

[NoGravitas]

Once you have a knowledge base connected to the language model, it's no longer a Stochastic Parrot, but something else entirely. The point of the paper is that simply continuing to scale up LLMs will not produce understanding, because a pure LLM has no connection between form and meaning. That link can provided in other ways, though (multimodal models, robot embodiment).

[dekhn]

But these language models are implicitly trained on knowledge by being fed large amounts of factual text, which (I presume) allows it to generate text that is factual (statistically more frequently than hallucinating nonfactual information). So probably recent models (which were being trained around the time the parrots paper came out) are really implict knowledge models already. Obviously they don't have embodiment, and it's still unclear to me what level of true embodiment in the actual, real, physical world is required to make these models more than just "parrots".