[svcrunch]

Take a look at the BEIR benchmark, which has served as one of the main drivers for development of neural IR systems since its introduction in 2020.

BM25 presents a challenging cross-domain benchmark, and it wasn't till ~2022 that neural methods overtook it. If memory serves, it was the sparse neural methods like Splade, although recent dense models can also beat it.

The caveat is that BEIR is suffering from overfitting at this point.