[orbifold]

The vaccine does unfortunately not work as well as hoped for in the case of the delta variant + a lot of people are still unvaccinated and delta is much more infectious. Cases and hospitalisations are also up in Israel https://graphics.reuters.com/world-coronavirus-tracker-and-m.... Here is a corresponding news item in Science: https://www.sciencemag.org/news/2021/08/grim-warning-israel-.... It also suggests that the protection against hospitalisation of the vaccine is not as good as it was hoped for among older people. From the publicly available data for Israel you can estimate the vaccine efficacy against hospitalisations to be ~59% (relative risk reduction).

[ithkuil]

Statistics are hard to interpret correctly. See https://en.m.wikipedia.org/wiki/Simpson%27s_paradox

[orbifold]

Well it is easy to work it out yourself. Israelis >12 years are 78/22 vaccinated/not vaccinated. Among the hospitalised 59/41 are vaccinated/not vaccinated. So you have 59/78 = 0.756 and 41/22 = 1.864. Therefore the ratio of risks is 0.756/1.864 = 0.41. Which gives you 0.59 as I mentioned. I believe that is the same way the original efficacy of the vaccine was calculated in the first place. Admittedly I have a PhD in Physics not statistics, so I might be missing something :).

[rsfern]

These calculations aren’t stratified by age / prior risk, so you can’t really infer the efficacy this simply.

[orbifold]

The original statements of vaccine efficacy were also not stratified in that way, were they? With delta you arrive at ~39% relative risk reduction of preventing a positive PCR (instead of >95%) and ~59% relative risk reduction of preventing hospitalisation. That would also be true if you randomly drew people from the population and conducted a study, no? Of course it still can mean that your individual benefit is much higher.

[ithkuil]

> so I might be missing something

You're surely calculating a useful number, but it's not the "vaccine efficacy". Definition from the CDC website:

    Vaccine efficacy/effectiveness is interpreted as the proportionate reduction in disease among the vaccinated group. So a VE of 90% indicates a 90% reduction in disease occurrence among the vaccinated group, or a 90% reduction from the number of cases you would expect if they have not been vaccinated.

Your calculation looks at the number of hospitalised people not in the people who got the disease.

So, on a first glance it seems you're computing a useful number: how effective is the vaccine from preventing your to get hospitalised assuming that you would have been hospitalised if you caught covid. Surely that's a useful number, it just shouldn't be confused with the >90% vaccine efficacy number, which measures a different thing.

Now, I do also have some qualms with your calculation of the "hospitalisation prevention efficacy" rate.

Let me rewrite your calculation symbolically:

    "Israelis >12 years are v/(100-v) vaccinated/not vaccinated. Among the hospitalised h/(100-h) are vaccinated/not vaccinated. So you have h/v = x and (100-h)/(100-v) = y Therefore the ratio of risks is x/y = z. Which gives you (1-z) as I mentioned."

Let's make a little thought experiment. Let's imagine that 98% of people were vaccinated with a vaccine with 90% efficacy. Since the efficacy is not 100%, some people will still get the disease and some will still get hospitalised. Since very few people in this scenario are unvaccinated, most of the people who end up in hospitals will be vaccinated. Let's imagine that 50% of those who get the disease get hospitalised. Since only 10% of those vaccinated that get infected contract the disease, only 5% of the vaccinated population will get hospitalized. OTOH (in this scenario) 50% of the infected unvaccinated people get hospitalized, but since only 2% of the people are unvaccinated, this means only 1% of the infected unvaccinated population get hospitalized; 99% of the hospitalized people are thus vaccinated.

    v = 98
    h = 99

    "People from our scenario are 98/2 vaccinated/not vaccinated. Among the hospitalised 99/1 are vaccinated/not vaccinated. So you have 99/98 = 1.01 and 1/2 = 0.5 Therefore the ratio of risks is 1.01/0.5 = 2.02. Which gives you -1.02 as I mentioned."

That formula doesn't seem to make sense.

[orbifold]

It does make sense from the numbers 99% of hospitalised vaccinated, 98% of the population vaccinated you should conclude that the vaccine increases the risk of hospitalisation. That is why the number comes out negative. It would be the same in https://en.wikipedia.org/wiki/Vaccine_efficacy if you swapped the numbers for vaccine / placebo. The number would come out negative if in a trial the vaccine worked worse than the placebo. In your scenario you confuse yourself by calculating with percentages, I think.

[ithkuil]

> you should conclude that the vaccine increases the risk of hospitalisation.

Not really. It just means that since in that scenario the vast majority of people are vaccinated the break-through cases (people who despite being vaccinated end up hospitalised) outnumbers the small percentage of unvaccinated people who end up hospitalised *despite* the fact that the vaccine efficacy rate remains constant.

I'm not sure if we're talking past each other, so let me take one step back and make another attempt:

I made an extreme example to make it easy to see. Let's make an even more extreme example without any numbers nor percentages:

Let's imagine that everyone, literally every single person in a population gets vaccinated. How many hospitalised people do you expect to see?

Since the vaccine is not perfect and some people get severely sick despite the vaccine, you'd expect to see some number of hospitalised people.

How many of those hospitalised people would be vaccinated?

Well, we just said that in this scenario everybody in the population has been vaccinated, thus everybody who got hospitalised is vaccinated!

Can we conclude that vaccines cause hospitalization? We clearly cannot conclude that from these numbers alone.

Can we agree on this before continuing to talk about which crucial metric we're ignoring in this discussion?

[orbifold]

I'm honestly not sure, maybe we are really just talking past each other. If there are no unvaccinated people left the relative risk reduction would be zero, that just follows from the definition. The absolute number of hospitalisations would still be meaningful. In the perfect world that everyone was vaccinated you would still see an age dependent decrease of effectiveness over time and relative to alpha. The real data from Israel seems to show that two vaccine doses in a very compliant population are not enough to effectively prevent the number of cases, hospitalisations and deaths to rise without additional other measures (boosters, masks, etc.).

[lamontcg]

The israeli data has issues because it was during the initial surge of delta which was predominantly in cities with high vaccination rates, which drives the VE down if you compare it against the population-wide ratio of vaccinated to unvaccinated.

Most studies of VE also do statistical modelling of how many of the unvaccinated have been infected and are actually recovered. This was the case of the CDC study from yesterday which had the headline number of only 66% efficacy against delta, but if you dig into the data they didn't test for antibodies or anything but modeled viral spread and had a 95% CI on that number of 26% to 84%.

That is literally garbage data.

The actual value could be closer to the higher end due to them underestimating community spread (pretty plausible) and due to selection effects where their unvaccinated population was higher risk for prior infection than they thought.

I thought that study might change my mind on the situation since they had done a much better job than the Israeli data on controlling for age, comorbidities, etc. But when you get down to it the VE data was still crap. And the mention of waning immunity seemed to be just tacked on with no supporting evidence other than the lower number VE number against delta.

Similarly, its now been found that the studies which equated similar Ct values to viral load are bad because there's less culturable virus in vaccinated individuals, and the Ct values drop faster which indicates vaccinated individuals are producing more viral debris at peak Ct and clearing the virus faster. So they would therefore be expected to have less transmission. Since they're staying out of the hospital and Ct values correlate somewhat with symptom severity that also suggests less transmission (the people who wind up hospitalized most likely transmit more before they get there).

Then there's that silly study out of China which found the odd result that viral-load-as-measured-by-Ct where 1,000 times higher against delta than against original Wuhan strain data from the pandemic when measured at the onset of symptomology (so not peak data, not average data, and compared against the earliest PCR results against the original pre-D614G virus). That quietly disappeared from scientific discussions, but the media keeps on citing it as the truth without any qualifications.

We know that vaccination reduced actual transmission and attack rates against Alpha and its very likely that it does the same against Delta. No data has contradicted that yet.

And I can't explain why so many scientists are so eager to undermine the message of vaccine efficacy. I guess they think it'll be easier to get vaccinated people to mask up than it is to get the unvaccinated to get vaccinated, and some of them are presumably just as addicted to bad news on facebook and twitter as the rest of the population is.

[orbifold]

Ok thanks, that is informative. From a policy perspective I think it is a bad idea to basically promise everything can return to normal after vaccination, if this appears to not have worked out in places like Israel. In Germany they are phasing out free antigen tests and they are not required for vaccinated. Both seem like really short-sighted policies.

[AndrewBissell]

It seems like lately there's been a big push for people to respond to the disappointing outcomes for the vaccines in places like Israel and Iceland with "Simpson's paradox!" or "base rate fallacy!" as a way of ignoring what the actual data is saying. The other day NPR had a segment about Iceland where they said "67% of cases being in vaccinated people might sound like a disappointing number, but you have to remember that 71% of the country is vaccinated" which still seems pretty underwhelming?

[orbifold]

What ultimately matters is if the hospitals can handle the load, delta is problematic because it produces new cases more quickly even among vaccinated and a sufficiently large number ends up in the hospital for it to become problematic. Both of these are demonstrated in Israel without invoking any kind of statistical analysis. Qualitatively the vaccine alone does not work well enough to lead to an R0 < 1, even when the most susceptible people are almost all vaccinated. I think that is super disappointing and at the same time I think it is really worrying that some politicians have not got that message in Germany / other places.

[lamontcg]

> Qualitatively the vaccine alone does not work well enough to lead to an R0 < 1

I honestly haven't seen any evidence of this.

With Alpha the attack rate for 80% of breakthrough infections was zero and the other 20% were only 1-3 cases. That's enough to drive the r0 down below 1.0 in a 100% vaccinated population. Nobody has done similar studies against delta breakthroughs.

[AndrewBissell]

Good thing we've had over 18 months since the start of this pandemic to add a significant number of hospital beds and nursing staff to help handle these spikes ...

[swiley]

They've been firing staff and closing beds.

[twooster]

Expanding on a sibling comment's reference to Simpson's paradox is the following analysis of the data:

https://www.covid-datascience.com/post/israeli-data-how-can-...

Basically, Israel is doing quite well even still. Yes, there's a reduction in efficacy, and yes there's breakthrough infections, but your risk of serious outcomes is still reduced by ~80%+, even in older populations _when correctly compared to unvaccinated older populations_.

[orbifold]

Isn't what you ultimately care about in terms of policy the effectiveness on a population level? Of course you hit diminishing returns, when you start vaccinating people that were not at risk in the first place. I'm not arguing against getting the vaccine btw., I'm vaccinated myself. It is just disappointing that it alone won't solve the problem.