[malf]

> Another critical decision is that they only investigated pairs of model-dataset with good initial performance.

groan

[bootsmann]

Overfit in training and then complain that it doesn't generalize well in production, a true classic.

[epups]

If this is what happened, what explains the fact that some of these models performed great for about a year before decaying?

[noobermin]

This seems rather dismissive for something actually published in a real journal and not just on arxiv for once, right?

[bootsmann]

If you check the abstract, then this got published more for describing these drift patterns and showing ways to visualize and detect them rather than dropping the weird statistic the article makes it about.