[scotty79]

16 seems like a suspiciously round number ... why not 17 or 13? ... is this just result of some bug in the code they used to do their science?

or is it just that 16 was arbitrarily chosen by them as close enough to the actual minimal number of dimensions necessary?

[woopsn]

It's a little arbitrary. Look at the graph on page 6, there's no steep gap in the spectrum there. 16 just about the balance point

[moi2388]

But there is a steep gap in the spectrum at 16 on page 7

[yorwba]

That's the spectrum of LoRAs, which are LoW RAnk by design.

[moi2388]

Yes. But from their paper: “In our analysis, we present compelling empirical evidence for the existence of universal subspaces within LoRA adapters across different modalities and tasks.”

I also don’t understand what they write under figure 2, since resnet50 has 50 layers, not 31.

[altairprime]

There’s lots of hockey stick charts in the paper that might answer this visually, if that’s of interest.