[kevsim]

I would also think it'd be an incredibly expensive way to train a model.

[ReptileMan]

Depends. I wonder what is the minimum reasonable amount of different tokens needed to lift up the weights.

[Jensson]

You store the output from ChatGPT, you don't run it again every time you do a training step. Generating millions of examples to add to your own training wont cost much at all relatively.