[scotty79]

Do you know of any research that tries to take large pre-trained model and make it smaller by cutting out least activated neurons and training it a bit not to loose performance?

[sebzim4500]

https://arxiv.org/pdf/2301.00774.pdf

[KRAKRISMOTT]

The entire field of ML distillation.