[luckyt]

You're absolutely right. A good way to tell if a paper is legit is search it on Google Scholar and see how many citations it has: this paper has none. If it's an arXiv preprint (not in a peer-reviewed journal) and also a low number of citations, definitely take it with a grain of salt.

[anjc]

This is clearly a bad way to tell if a paper is legit, seeing as this particular paper seems to have been published a week ago and will therefore probably not have any citations for close to a year.

Judge papers based on their methodology, evaluations and findings (at least).

For some fields, you'll find the most cutting edge research on arXiv.

[luckyt]

Yes, ideally you would carefully read the paper and evaluate its methodology, evaluations, and results, but this requires you to be knowledgeable in the field. That's why peer review exists -- some experts in the field read it and determine if it passes the bar, so you don't have to.

Citation count is useful for determining the relative importance of published papers. 10-100 citations is ok, 100+ citations means it made a significant contribution, 1000+ citations means it's a landmark paper in the field and probably worth reading. In the case of this paper, it's very new so citation count is not very meaningful, but it's a useful heuristic to evaluate most papers.

[anjc]

A lot of 'gaming the system' and backscratching goes on in Academia. Relying on citation counts, journal/conference names, and the oversight of 2 or 3 peers (who may be 1st year PhD students) is a veerrrrry dubious heuristic. Having said that, it'll certainly cut down the size of your reading list.

[Someone]

https://www.sciencedirect.com/science/article/pii/0022072889... has been cited 603 times, so it is somewhere between “made a significant contribution” and “landmark paper in the field”?