|
|
|
|
|
|
by whimsicalism
1164 days ago
|
|
|
As someone else mentioned [0], the procedure would basically be to train a DyLoRA for an initial few iterations, then do a search among the layers to find the best scoring combination of ranks, and then train pruned to just use those ranks to completion. Seems complicated but I could see it being useful potentially. [0]: https://news.ycombinator.com/item?id=35517353 |
|
|