[seydor]

Can rank decomposition be used to reduce the original weight matrices as well? Or are they assumed to be compressed already?

[metanonsense]

Those fully trained networks are usually considered full-rank. At least that is what they say in the LoRA paper.