[leedrake5]

> They found that the model relied on the style of language used in papers to judge if a study was replicable.

I think the failure here is to train the model on published results. I’ve worked with scientists who could write and infer a marvelous amount of information from shit data. And I’ve worked with scientists who poorly described ingenious methods with quality data. The current academic system incentivizes sensationalization of incremental advances which confirm previously published work. I’m not in the least surprised at the manuscript level that replication would fail.

The proper way to do this would be to log experimental parameters in a systematic reporting method specific to each field. With standardized presentation of parameters I suspect replicability would improve. But this would present an near impossible degree of coordination between different research groups. But it would be feasible for the NIH or NSF to demand such standardized logging as a condition of grant awards of a certain size.