|
|
|
|
|
|
by redox99
993 days ago
|
|
|
> I have not yet seen it successfully done, and I suspect that updating fractions (~.1%) of the original weights with PEFT methods won’t help. Nitpick, but although when training LoRAs you're only training 1% or less (depending on rank) of the number of parameters of the entire model, the adapters affect the entire model and after merging the LoRA all of the weights of the model are updated. |
|
|