[monocasa]

Or just perform a form of distillation, where you don't actually change the hyperparameters, but maybe shift around the embeddings or something.

You could even have another model watch the distillation process to check for goofy backdoors (which is about the best you're going to be able to do since detection of backdoors is np hard IIRC).