[barmstrong]

This is a great summary of how to show causal effects. Thx for pointing it out.

I feel like we need a new way to do peer review, that is more real time - so that papers can be upvoted/downvoted, flaws can be pointed out - and we have some way to assess the truthiness of what the paper is claiming. Your comment is a step in this direction (but we're not capturing the wisdom of the crowds quantitatively around papers today - arxiv is great, but 1990's era web design).

I'm working with a team that is trying to build better tools for science at https://www.researchhub.com/about

Rethinking peer review is one item on the roadmap.

[thomasahle]

> we need a new way to do peer review ... so that papers can be upvoted/downvoted

This one managed to get 700 upvotes on HN and still be meaningless.

[Melting_Harps]

> I feel like we need a new way to do peer review, that is more real time - so that papers can be upvoted/downvoted, flaws can be pointed out - and we have some way to assess the truthiness of what the paper is claiming. Your comment is a step in this direction (but we're not capturing the wisdom of the crowds quantitatively around papers today - arxiv is great, but 1990's era web design).

> This one managed to get 700 upvotes on HN and still be meaningless.

Agreed. This is why I don't think voting with online peramters, like down voting, will ever be viable choice; because aside from things like Cybil attacks, the reality is that Social Media has normalized and acutely optimized group think and the tribalism that often follow. For those of that don't engage in it its incredibly alarming and disconcerting how many succumb to it's practices in real Life.

What I will propose as an alternative is something that was first tried with Andreas Antonpolus' first book 'Mastering Bitcoin' wherein the chapters are each individually uploaded to Github and follows the same process that OSS does, wherein commits and corrections are submitted to alter the books ultimate maintained version, and never really being fully 'finished' and can be amended and annotated as needed to suit the updates that follow (eg: Segwit or Lightening Network) or in this case perhaps a replicated experiment that provides a larger sample size. Replication in Academic Papers is nearly non-existant, despite the notion that Peer Reviewed (especially in STEM) was to be a the critical component that made it invaluable.

This could also effectively undo the walled-garden-extrotionist model that afflicts Academia's Peer Reviewed model and foster more interactive and International work without being present in a University. Samples or specimens may need to be more tightly controlled and transported, but this could effectively be done over night if the Will exists for little to no money in anything other than training.

If the International University model is undergoing mass disruption and shifting toward a mainly online learning platform, this too could help mitigate these glaring problems and have a Global Repository for ongoing research.

Its almost stupid not to do it at this point given the MANY pitfalls that its current model forces down our throats.

[chrismatei]

I agree with this.

I'm currently doing research, and one factor that always makes go nuts was the actual time to go and find an article and then donwloadit. I can say that I have used or rather abused sci-hub so that I can find related papers.

Having something like "github" style + the option to visually view the changes updates, what are the connection, who did what, where is coming fro, something in the way the site conectedpapers does is also very powerfull.

Lastly, the only concern in here it is that if it's ever leving document how ca you asses as whole one document if its good quality at certain given time ?

So my proposal wil follow under "feature request" type where even if the changes are made, having the ability to read the "metadata" or quality check done by lets say AI(plagarims, etc, etc,) + with the feedback from online reviewer = qood quality check.

[thomasahle]

> commits and corrections are submitted to alter the books

I think this is good idea fro books, and I wish more scholars would contribute to Wikipedia and other peer editable overviews.

I'm not sure quite how it would work for new results though, for which it is still not quite clear how they fit into the current knowledge, or if they are worthwhile at all.

People also tend to feel really strongly about their own original ideas, and may try to push them forward when they should rather be forgotten.

I don't know how to prevent something like that, other than having a BDFL or individual publishing (as in the current system)

[hunter-gatherer]

Gitlab has an article on their website somewhere where they address some of your concerns. I'm on mobile now and can't find it, but the article advocates for a 'Handbook first' approach to documentation. We did this at my last job and I found it well worth the effort. The BDFL ended up being the quality of our work. In other words, the most correct version always wins out.

[setgree]

> This one managed to get 700 upvotes on HN and still be meaningless.

On the other hand, the top/most salient comments on it are all pretty critical of its causal claims.

[on_and_off]

Not to mention that it also needs to account for science advancing one funeral at a time.

How to distinguish between real flaws or just a good theory that is not yet taken seriously ?

[setgree]

I just uploaded the papers I’ve co-authored — all were licensed CC 4.0 — to Research Hub and emailed to say hello. Looks like a great platform!