[loxias]

Hi! I in _no way_ mean to detract or malign or "anything negative" the parent comment (communication is hard!!), BUT I really compliment that exact sentence. :)

My background contains signal processing, "pre-deep learning ML", systems engineering, and firmware, and that sentence jumped out at me as crystal clear in my mind, despite not knowing what HuggingFace is or PyTorch.

Correct me if I'm wrong: These huge models involve lots of weights used in large matrices. The contribution of this work is to plug in some matrix factorization and learn a lower dimensional representation, instead of a large second matrix.

Fantastic!

Also makes me wonder what other performance improvements await through proper application of established and well known Mathematics. :D