[Nevermark]

I don't think we would have AI slop, if they were just trained on high quality media.

Less relatable AI, maybe, but not sloppy AI.

AI trained on other AI (i.e. distilling), can be better than the AI they were trained on, likely because of this reason. Each iteration can raise the bar.

As apposed to the deep spiral of generating shiny attention crystals for humans, and then training on that.

[inigyou]

The slop isn't a property of the LLM but the workflow it's utilized in. Just now I asked ChatGPT about the linguistic effect where a CRT could be described as a giant flatscreen, a landline could be described as a wired cellphone, or the original Rogue could be described as a roguelike. Unfortunately it doesn't know what that's called either, but that is a non-slop use of an LLM, like a semantic search engine. On the other hand if you print LLM output into a book it's almost certainly slop no matter how the model was trained.

[dncornholio]

Training AI on AI will most likely increase slop. LLM's add randomness in their outputs, books usually don't.

[Nevermark]

That isn't correct.

Obviously, you can handicap any model with an indiscriminate dataset. Are you training on slop? Why?

More data is not automatically better data.

Just three (of many) advantages of quality data selection: (1) It gives a clearer signal. (2) It, ironically, reduces the complexity of what needs to be learned, since slop adds its own complexity. (3) It reduces dataset size which means more learning per watt (and per just about any other cost). The advantages compound.

Humans are no different. Children surprise with their ability to absorb sophisticated relationships and skills, while people who had bad examples struggle to recover.

99% of human effective intelligence is higher quality cultural knowledge learned as children. None of us had to spend centuries deciding whether zero, negative numbers, the square root of -1 are numbers. Slop + quality is not quality.