[passion__desire]

This won't be necessary in future AIs. As AIs will start aligning tokens from all the rich modalities of audio, video, 3D with text so that they can express complex ideas, they will bootstrap in proper language generation.

I don't think college essays, etc would contain anything novel. Future techniques could smoothly interpolate better creating ever-anew wordmud.

[nicklecompte]

I agree with your overall point that an AI which can learn about the world directly won't need eleventy billion documents to learn language generation. Just two comments:

1) Based on how pre-verbal children learn, one nitpick is that I strongly suspect we need to give AI touch and a sense of space in order to truly understand quantity, causality, object permanence, etc.

2) Something that is not a nitpick: even a superhuman multimodal AI wouldn't have direct access to human emotions, sexuality, ideas of natural beauty, etc. I don't think humans have run out of interesting things to say about these ideas.

(In particular, I don't think a superhuman AI is capable of understanding music unless it is directly emulating the biological processes by which humans understand music. The issue is not "logical" - melodies don't actually make sense analytically.)

[dleink]

> I don't think [things created by humans] would contain anything novel.

That's quite a proposition.

[passion__desire]

Not every essay is created equal. Plus I don't understand what would a new way of combining same words, given llms already have seen trillions of tokens, would achieve. llms could inpaint to arrive at similar texts.