|
|
|
|
|
|
by HarHarVeryFunny
30 days ago
|
|
|
Yeah, and they are not self-distilling in the sense of training on the outputs of the model, but instead operating on the pre-output probability distribution - trying to minimize teacher-student differences. Still, at the end of the day this is a type of SFT, not continual learning in the human/animal sense. |
|
|