[brookst]

The best was the Ted Chiang article making numerous category errors and forest/trees mistakes in arguing that LLMs just store lossy copies of their training data. It was well-written, plausible, and so very incorrect.

[supriyo-biswas]

Neural network based compression algorithms[1] are a thing, so I believe Ted Chiang's assessment is right. Memorization (albeit lossy) is also how the human brain works and develops reasoning[2].

[1] https://bellard.org/nncp/

[2] https://www.pearlleff.com/in-praise-of-memorization

[brookst]

The fact that some neural network architectures can compress data does not mean that data compression is the only thing any neural network can do.

It’s like saying that GPUs can render games, so GPT is a game because it uses GPU.

[dr_dshiv]

I felt the same way. But I’d love to read a specific critique. Have you seen one?

[elefanten]

Here’s one from a researcher (which also links to another), though I’m not qualified to assess it’s content in depth.

https://twitter.com/raphaelmilliere/status/16240731504754319...