[ahoho]

Color me skeptical on BERTopic. Without human validation, I'm not convinced that it's an improvement over existing methods.

I'm an author on a recent paper about automated topic model evaluation [1], and we found that current metrics do not line up with human judgements as well as previously thought. To my knowledge, BERTopic has only been evaluated on these automated metrics.

For datasets of under a few hundred thousand documents, Mallet (LDA estimated with Gibbs sampling) can produce stable, high-quality outputs in minutes on a laptop [2]. Even larger datasets remain tractable, although depending on your use case you may be better off subsampling.

It's possible that I've missed something, but I'm not clear on what benefits BERTopic has that existing methods do not. I don't mean to be overly negative---it has a nice API and the approach seems reasonable---I'm just wondering what's really new here.

[1]: https://proceedings.neurips.cc/paper/2021/hash/0f83556a305d7... [2]: https://mimno.github.io/Mallet/ [3]: https://maartengr.github.io/BERTopic/faq.html#why-are-the-re...