[cooljoseph]

You can also have another model "mentor" a new model you are teaching to speed up training. You don't have to start from scratch with zero knowledge. This is done a lot in what are called distillations.

[eru]

You can also re-use a lot of the infrastructure. Eg you can re-use your training data.

[fnordpiglet]

This came out a little bit ago, my open question is if this approach can be used to port weights between architectures like this.

https://arxiv.org/abs/2402.13144