[ijustlovemath]

This is the bit I've never understood about training AI on its own output; won't you just regress to the mean?

[astrange]

It's not trained on its own output. You can generate infinite correctly worked out math traces and train on those.