[xioxox]

I don't think comments would be useful. To make a proper comment on a paper requires a significant amount of work (as does refereeing). Lowering the barrier to entry would produce some first impressions, the usual moans about lack of citation and misunderstandings. If a paper deserves a serious comment, I think the proper format for that is another paper.

Although citations are horrible metrics, I think some sort of reputation ratings based on quick likes would be even worse and even easier to game.

[return0]

>To make a proper comment on a paper requires a significant amount of work (as does refereeing).

Let's not fool ourselves, not all refereed papers are great, not all of them are significant and not all of them are published in high impact journals. Most of the reviewing is cursory, menial, and often delegated to postdocs/phds. Most scientific topics are highly specialized, and you 're unlikely to see trolls bothering to comment on them. Writing another paper as response is not a solution either: the pace of article publishing is months and years, not seconds.

An example: I have often found the discussion of scientific papers here, in HN, to be illuminating, clarifying or countering issues, and offering a wider perspective that is often not mentioned in the article itself. And it's not like everyone in HN is a luminary, just mostly inquisitive people. I also see articles discussed in a useful way in twitter. Why can't we open up this discussion?

I don't think comments are to be taken as reputation metrics either. But i do think a simple, helpful open discussion section is missing from every paper that i have read. I believe the main barrier to it is that academics do not want to get off their high horse of untouchability.

To be precise, many journals do have a comment section, but nobody uses it.

[JD557]

>To be precise, many journals do have a comment section, but nobody uses it.

GitXiv[1] also has a comment section and voting features, and nobody uses it.

I know it has a small user base compared to ArXiV, but I think that its format resembles what ArXiv might look like with comments, votes and reproducible code.

[1]: http://gitxiv.com/

[jessriedel]

> GitXiv[1] also has a comment section and voting features, and nobody uses it.

Likewise SciRate is a reasonably popular arXiv overlay, but the comments are never used: https://scirate.com/

[egjerlow]

> If a paper deserves a serious comment, I think the proper format for that is another paper.

That's the thing, though - speaking as a former academic, the reaction when you see a paper that needs serious commenting is mostly "I'll just ignore this paper, since it has these issues", and you continue working on your own stuff - which again is because of the need for citations. If comments had larger effects on your career, they would be worthwhile making. I would guess the effect would be less papers and more 'collaborative science' where you start from a paper and either suggest improvements or improve it / replicate it yourself. I don't see the current proliferation of papers to be a good thing.

As for the worry that there will be too many 'laymen' doing the commenting, there could (as suggested elsewhere) be a verification process - i.e. there could be a 'Verified Ph.D.' (or some other measure of merit) comment section in addition to the 'layman' section.

There could also be a 'replication' section, where replication efforts will be rewarded - possibly by giving the replications some fraction of the paper's total 'reputation'.

In short: If comments are made to really count for scientists, I think there will be a more healthy scientific process going on. Viewing a paper as an evolving thing is IMO more in line with how science does work anyway - a result should be replicated before it's accepted, which is not the case now.

[dalke]

We need only to look to journals to see counter-examples to your proposition that "If a paper deserves a serious comment, I think the proper format for that is another paper."

For example, a journal might have a "Letters to the Editor" section for people to voice a serious comment, even though such a letter is not "another paper".

[Aeolun]

The only thing you have to do to make comments worthwhile is restrict them to actual academics. Even if they went on to post shit, it would only destroy their own reputation.

[adrianratnapala]

This destroys one of the potential benefits of comments -- which is to open up what might be closed communities. Academic disciplines can at times turn into echo chambers.

But there is the rub: if you make comments open, then you will will probably be flooded with. If you close them up, then fail to get the benefits of openness.

[mjohn]

There are plenty of professionals who would be able to contribute, but are not academics affiliated with a university.