[make3]

Alpaca uses knowledge distillation (it's trained on outputs from OpenAI models). It's something to keep in mind. You're teaching your model to copy an other model's outputs.

[thewataccount]

> You're teaching your model to copy an other model's outputs.

Which itself was trained on human outputs to do the same thing.

Very soon it will be full Ouroboros as humans use the model's output to finetune themselves.

[visarga]

> You're teaching your model to copy an other model's outputs.

That's a time honoured tradition in ML, invented by the father of the field himself, Geoffrey Hinton, in 2015.

> Distilling the Knowledge in a Neural Network

https://arxiv.org/abs/1503.02531