[lyso]

So it looks like with this method, if a journal publishes more papers, this will give it more of a chance to boost its h5-index? This probably accounts for the high level of arXiv, and PLoS One beating out PLoS Biol.

One problem with impact factors is the way that a few articles can account for the majority of citations. For instance, a bioinformatics method that is widely used could attract thousands of citations, boosting the impact factor of the journal by a few points. This method doesn't solve this, as it expressly focuses on the top n articles and ignores the impact of the remainder. For instance, PLoS One's score of 100 is because the top 100 articles got 100 citations - it says nothing about the distribution of the rest.

[mjn]

That does seem to be the case. I'd be interested in some kind of median-citedness measure as well, to distinguish a venue that publishes 100 high-impact articles a year, from a venue that publishes 10,000 articles a year, of which 100 are high-impact.

In particular, it's not robust to one factor often mentioned in the bibliometrics literature, trivial changes in agglomeration size. Say a set of 200 articles are published by either: 1) a single journal; or 2) two journals, which publish 100 of them each. In each of the hypotheticals, individual articles have the same citation counts. Under this metric, #1 gets a higher ranking, meaning that you can raise rankings without increasing paper quality by just agglomerating journals. (You can even run the two former journals separately inside the new journal if you want, with a two-track review structure, as long as there's only one title on the front page.)