[penagwin]

The problem is that there is so many weights in the model that they don't fit in memory. You can lower the number of weights, which will lower the effectiveness of the model.

The thing is that when you're going for leaderboards you're reaching for every last percentage point, so the efficiency of the model size/performance isn't a concern, you want to ramp up the resource usage to as you have access to.

TL;DR - Yeah basically most people will run a "slimmed down" version of the model that isn't "as" performant, but is still an improvement over previous models and actually fits on your machine.