[kriro]

I think that's a pretty great idea actually. If anyone here has the reach to get this on their table please do reach out. It would be even greater if they'd use some replication metric in their ranking algorithm for the papers.

The danger however is that this would just lead to only me-too studies being accepted and failed replications still being rejected.

I'd also love if they'd add (possibly Google hosted) repositories where the data/scripts etc. that belong to the paper can be uploaded and archived for all eternity.

[mwambua]

I saw something like this a while ago... Codalab (http://codalab.org/). It's run by Percy Liang (https://cs.stanford.edu/~pliang/). It lets you create executable worksheets to go with your papers; and people can validate both the code and data used in your experiments.

[chronic81]

http://gitxiv.com is far superior than Codalab. Source: I go to Stanford.

[kriro]

Looks pretty cool indeed, bookmarked. However I believe it is focused on Computer Science. Nothing wrong with that but I think they key is convincing scientists that are not exposed to ideas like version control, open source (and I'd argue data sharing due to the rise of data mining) by default (or are less computer savvy in general).

[mwambua]

Interesting... It seems broken right now; but the github & arxiv integration sounds pretty cool, so I'll keep an eye on it.