[swyx]

do LoRAs conflict with your distillation?

[sangwulee]

The architecture is the same so we found that some LoRAs work out-of-the box, but some LoRAs don't. In those cases, I would expect people to re-run their LoRA finetuning with the trainer they've used.