Hacker News new | ask | show | jobs
by harry8 1737 days ago

    we tried using 0.10 mL, it didn't work
    we tried using 0.11 mL, it didn't work
    we tried using 0.13 mL, it didn't work
    we tried using 0.15 mL, it didn't work
    we tried using 0.17 mL, it didn't work
    we tried using 0.16 mL, it didn't work
    we tried using 0.18 mL, it didn't work
    we tried using 0.20 mL, it didn't work
    we tried using 0.14 mL, it didn't work
    we tried using 0.12 mL, it worked so we published
Do you want to know the ones that "didn't work" existed? Or are you happy with just the one that "worked" being written up in isolation?
2 comments

Especially for small effect size and suppressing what didn't work, this is one obvious way of many to perform p-hacking for publication acceptance.

https://en.wikipedia.org/wiki/Replication_crisis

i don’t want to know about each test that didn’t work as a separate publication, that’s for sure!
That's true, we would need a way to collect this data so it's easily seen as part of a whole.

E.g. if you search for eggs and cholesterol you should find all studies with their summarized results on whether eggs are ok or not for your cholesterol, grouped by researcher, so if somebody does 200 studies to find the one positive it's instantly visible.

You would read a meta-study that summarizes those tests - especially because they might potentially made by different labs, and the fact that one of them worked might be actual a real effect caused by some other difference in the experiment.