[lxe]

Kudos for the authors for providing the code https://github.com/huawei-noah/KD-NLP/tree/main/DyLoRA and the roberta example. Considering the current state of the OSS LLM community, I'm guessing someone is already porting it to Llama and gpt-style models.

[kernelsanderz]

Adding this to the huggingface peft library would be amazing. That's the main library that people using LoRA are currently using. https://github.com/huggingface/peft/issues/289

[brucethemoose2]

The stable diffusion community has, unfortunately, largely ignored peft because the training/inference scripts largely ignored diffusers.