[lottin]

> Bayesian inference assumes the observed data are fixed and aims to quantify the evidentiary support for all possible levels of treatment effectiveness based on the data at hand.

The problem with this approach is that we can only observe ONE level of treatment effectiveness, i.e., the level of treatment effectiveness that the treatment actually possesses. All other possible levels of effectiveness are entirely hypothetical. There's no data about all these other possible levels of effectiveness because they don't occur in reality. So the data cannot possibly tell you anything about how likely is the observed outcome, because the observed outcome is the only outcome that you observe. I

This criticism was made over 100 years ago, and Bayesians still don't have an answer. They just keep going as if nothing happened, but the reality is their methodology is utterly and fatally flawed.

[ComplexSystems]

> So the data cannot possibly tell you anything about how likely is the observed outcome, because the observed outcome is the only outcome that you observe.

This could also be viewed as supporting the Bayesian perspective, where the observed data are not viewed as random variables - they are fixed. This is because, as you say, the observed outcome is the only outcome that you observe. It is the classical setting, in comparison, where we instead do our analysis by treating the sample as a random variable, placing the counterfactual on other non-observed values ("what if I had drawn a different sample?"), even though we didn't. Bayesian methods treat the data as gospel truth, and place the counterfactual on the different parameters ("what if the population were different?"), even though it isn't.

The other criticism you have is

> The problem with this approach is that we can only observe ONE level of treatment effectiveness, i.e., the level of treatment effectiveness that the treatment actually possesses. All other possible levels of effectiveness are entirely hypothetical.

This is true of both Bayesian and classical methods. We build models that would explain how different hypothetical levels of effectiveness would affect what data we should expect to see - that is the whole point. Classical methods also involve exploring scenarios in which purely hypothetical values of the parameter may be potentially true, and characterizing counterfactual samples that could have been drawn from them, even though in real life they couldn't have been.

[lottin]

Statistical inference is based on random sampling. The data has to be random, otherwise it doesn't work.

I wrote another comment here clarifying my point, if you're interested: https://news.ycombinator.com/item?id=47566033

[NeutralCrane]

It’s hard to understand what criticism you are making, or what alternative this criticism doesn’t apply to, in contrast. Would you care to elaborate?

[lottin]

Imagine we want to know the ratio of men to women in a particular population. We could count all men and women one by one, but it would take too long, so instead we take a random sample and count the men and women in the sample, and from that we infer the quantity that we want to know. This is statistical inference.

In Bayesian inference, the population ratio is seen as a quantity that can take different values each with a associated probability (i.e. a random variable), and the result of Bayesian inference is an estimate of the probability distribution of the population parameter, in this case the population ratio. Now, in reality the population ratio is a concrete number, say 9-to-10, meaning that there are 9 men for every 10 women in the population. But Bayesians don't care. They'll tell you that the population ratio is a random variable which can take many values, and that the probability that it is equal to 9-to-10 is whatever number between 0 and 100%.

This is nonsense because the population ratio is NOT a random variable. People don't come in and out of existence randomly, right? In a way, they're saying there are infinitely many possible universes, each with a different population ratio, and then they come up with an estimate of the probability that the universe in which the ratio is 9-to-10 has whatever probability of occurring. This is absolutely BIZARRE. (I hope you agree). And it's wrong because it's impossible to know how likely one universe is compared to all other possible universes, since we live in our universe and this is all we can hope to observe.