[nl]

Hmm maybe. Turns out the author trained a logistic-regression classifier on the embeddings too, but didn't report the results:

https://github.com/thelgevold/fine-tuned-classifier/blob/mai...

[dev-experiments]

Expanding on this experiment using logistic regression is an interesting continuation, detailed here: https://www.teachmecoolstuff.com/viewarticle/using-logistic-...

In summary: Using logistic regression actually improves accuracy, but also performance during both runtime and during training.

[nl]

Is that overfitted?

Do 5-fold cross validation, maybe stratified.

[dev-experiments]

It's the same dataset as the original experiment, but could be interesting to dig into it more.