[philmcp]

This is fantastic progress, great to see

16x smaller = 41.5GB though

More research needs to be undertaken in model compression imo

[riku_iki]

On superglue benchmark, much smaller Deberta outperforms vanilla T5: https://super.gluebenchmark.com/leaderboard

I am curious why authors preferred T5?..

[danielcampos93]

T5 has a notion of prompting. None of the *BERT have notions of prompting.