[PaulHoule]

I use the 420MB mpnet model. They finally have a T5 that gains a point of performance but is almost 10GB. The SBERT folks already think most people would be happier with a model smaller than mpnet but faster.

I have tried fine-tuning other BERTs and not had one that was really worth using. One of these days I want to train a T5 to do something kinda generative like putting Mastodon tags on articles.