Hacker News new | ask | show | jobs
by quantgenius 1880 days ago
If the data are so good, why hasn't the vaccine been approved. Mainly because it's not clear that you could replicate these results, partly because of how the data was analyzed, partly because of how the experiment was run, partly because AZ has (and openly admitted) been p-hacking their results.

If you are looking for 95% significance and you test something ineffective 20 times, you should expect to get one significant result.

AZ and the Oxford team openly did this this prior to this trial and actually admitted to it in a press conference. In fact they not only released data only from some trials but they only released some of the data from some of the trials.

In the real world, where you look at results from any trial in the context of other trials and available data, these results are not credible.

The AZ, PFE and MRNA vaccines were designed to combat the original virus. PFE and MRNA were over 95% effective in trials where they were tested against only the original strain but only about 90% effective against a mix of the new strains and the old strain that exists now. PFE and MRNA and all vaccines in general are somewhat less effective in over 65s. AZ was only 65% effective against the original strain in a population which included no over 65s in the original trials and the data from these trials were p-hacked and so almost certainly overstated effectiveness. AZ would have you believe that their vaccine is now magically 10% more effective in an environment where the variants are circulating and the 20% of the tested population is over 65. This when, real-world large scale population level data in populations where it's been widely used shows an effectiveness nowhere near that.

Also remember, this is in a context where they have openly, in a press conference admitted to p-hacking. The head of the Oxford team actually, for real, insisted that if the vaccine efficacy is seems significant in a sub-sample of the tested population at any point before the trial endpoint, it's effective even if it's not significant in the entire population at any point or not significant at the endpoint. There is something like a 70% chance that if you used his criteria, a saline injection would seem to be effective.