[seydor]

The models will become much smaller, there are already some papers that show promising results with pruned models

And transformers are not even the final model , who knows what will come next