[robbedpeter]

Smaller models with better performance are beginning to arrive. Things like RETRO, better training data, longer training time, and scale optimization will have these models on phones and desktops doing crazy things in the near future.

[teruakohatu]

They are but performance is decreased. In many cases transformers are encoding vast amounts of training data within the insane number of parameters.