Hacker News new | ask | show | jobs
by CrampusDestrus 1062 days ago
it's 2023, we have the means to cheaply record and store audio and video evidence for basically any medical experiment. we can record every patient reaction and opinion without relying on the reasearchers' hearsay. we also have the means to store and distribute all the binary/textual raw data gathered throughout the experiments.

maybe as an intermediate step we could make available all the recordings to the peer reviewers and only offer the raw experimental data bundled in the paper publicly? maybe in the future we can have 1TB studies without breaking a sweat? maybe all the money we give to publishers can be spent on servers to archive all the primary data so at least we aren't simply filling the pockets of MBAs?
5 comments
WastingMyTime89 1062 days ago
> maybe as an intermediate step we could make available all the recordings to the peer reviewers

The issue is clearly not the amount of data available to peer reviewers considering it's already easy to detect major flaws in a quarter of published peer reviewed research. The issue is that peer reviewers do a shoddy job which should surprise no one having ever published peer reviewed research.

And to be fair why should they do better? It's generally unpaid, it's poorly paid when it is paid and it's not particularly well considered.

link
sonicshadow 1062 days ago
Sounds like a YC idea?
link
pbmonster 1062 days ago
These are medical trials. How do you preserve the patients privacy in all of this?

Or do subjects need to wave all their doctor-patient privacy rights before joining any trial?

link
CrampusDestrus 1062 days ago
If we discover that we can´t trust researchers then what else are we left with? Doctor-patient privacy works if the doctor is truthful in their reporting
link
Slaminerag 1062 days ago
It's generally permitted to share de-identified patient data. As long as you're not sharing patients names, medical record numbers, birthdays, and a couple of other fields, you should be fine.
link
lost_tourist 1062 days ago
Maybe we could do a double blind (including scientists) study where everyone waived their rights & are recorded then in another "typical" conditions study do none of that and compare the two and see which one seems to have the best, most accurate results.
link
ttpphd 1062 days ago
LOL looks like someone has not had to get data collection protocols through IRB approval...
link
CrampusDestrus 1062 days ago
if researchers are so untrustworthy then what's your solution?
link
Balgair 1062 days ago
For a 'quick' overview into the mess that is IRBs, this book review is a good starting place: https://astralcodexten.substack.com/p/book-review-from-overs...

TL;DR: IRBs are a mess, hyper-individualized, and the problem ain't getting any better any time soon.

link
dennis_jeeves1 1062 days ago
>it's 2023, we have the means to cheaply record and store audio and video evidence for basically any medical experiment. we can record every patient reaction and opinion without relying on the reasearchers' hearsay. we also have the means to store and distribute all the binary/textual raw data gathered throughout the experiments

This is a great approach IMO. Additionally skeptics (for example anti-vaxxers), should be physically present at the trials.

link
jamesdwilson 1062 days ago
how do you prevent cherry picking?
link
CrampusDestrus 1062 days ago
that's such a vague question

for example, if you have 50 partecipants but only provide the multimedia evidence for 20 of them your study should be thrown out the window

link
dolni 1062 days ago
What is preventing someone from having 200 participants, but saying they only had 100 participants, and then only providing evidence for 100 participants?
link
throw9away6 1062 days ago
You have to declare the study population before you request the next round of funding. Thereby fixing the problem.
link
CrampusDestrus 1062 days ago
The researchers will have to have gone through some kind of third party agency to get the partecipants. This agency should be queried to see the number they report
link
dolni 1062 days ago
How does this agency determine who can meaningfully participate in the study? Are they going to have the expertise to make that determination for _every_ study that could possibly be conducted?

What is the difference (to a layperson) between cherry-picking participants and rejecting participants because they do not meet your study's criteria?

Who funds this agency?

Do the members of the reviewing agency have their own biases, and might those biases tarnish the reputation of a study that is actually well-conducted? (hint: this already happens in journals)

link
linuxftw 1062 days ago
Nothing. Pfizer did this, albeit wasn't 50% of the enrollment.

You can see that here: https://kirschsubstack.com/p/pfizer-phase-3-clinical-trial-f...

link
lesuorac 1062 days ago
No Pfizer did not do that.

Pfizer had a trial with 21823 people in the Expirement group and 21823 people in the Placebo group. In the results they excluded data for 1790 from the Expirement group and 1585 Placebo group. However, _crucially_ Pfizer never claimed there were only 100 people in the study after starting with 200; you know pfizer excluded 3375 people because _Pfizer told you_.

link
raverbashing 1062 days ago
"But we have 20, the other 30 volunteers were removed by unrelated reasons" (and it is common to exclude volunteers from experiments)
link
jonlucc 1062 days ago
Every clinical trial paper I've read has a discussion of inclusion and exclusion criteria. I think for the trial to be registered, it has to include this information.
link
CrampusDestrus 1062 days ago
"unrelated reasons" should not be an acceptable excuse though. either state the reason or it goes into the trash. and if they were private reasons you can still contact them to confirm they left on their own volition and/or they didn't finish the trial without getting into specifics. you only need one lie to suspect the whole thing
link