Hacker News new | ask | show | jobs
by andrewla 2658 days ago
From [2], above: (hazard ratio [HR] 0.71, 95% confidence interval [CI] 0.50–0.998, P = .049)

Yowzers -- just squeaked by on that one, didn't they. Given the unknowns around dropouts, I would be frankly astonished if these results stood up. It mainly looks like the "speed" category dubiously "benefited" from a higher death rate; the ~20 excess deaths in that subset can neatly account for ~2 additional cases that drives significance to random-chance levels again.

In addition, the meta-analysis seems to indicate that only memory-based interventions work, while the individual study linked showed differently. Hopefully as pre-registration becomes mandatory it should become clearer how many of these studies were abandoned because of null results.

2 comments

Can you recommend a book or course that can help me understand stats on clinical trials (if not a stats book with emphasis on clinical trials)?

I feel like I am of those that can easily be fooled by misleading claims.

From [1] (the meta-analysis), "Working memory interventions proved most effective (g = 0.479), though memory, processing speed, and multi-domain interventions also significantly improved cognition."

From [2] (the crash results), deaths and drop-outs were handled by adjusting crash rates to total person-year exposure times. In this study as a whole, non-completion rates were not significantly different across groups.

Yes, pre-registraton is an excellent idea.