[Fomite]

Find me where I said proof.

There are all kinds of things that we cannot prove, because it is either impossible or wildly unethical to conduct a randomized study. For those things, you can make a determined effort to control for as many "third factors" - the technical term for them is confounders - and that gives you a level of evidence which is well above guessing.

Since I can't reply to your comment, my responses here:

> "You didn't say proof but you said it's better than guessing, and I don't agree with you at all."

It is better than guessing. You're welcome to disagree, but a well conducted observational study is considerably firmer evidence than pulling it from your posterior.

> "What if there is a correlation between Vegetarian-lifestyle and Serial-killers ? Does it tell you that it's better than guessing ? Do you even question if the association/correlation makes remote sense ? Is there any underlying mechanism of action that would remotely explain rationally why this correlation could be linked to any real causation phenomenon ?"

All you've done is describe a really bad study. You can have really bad RCTs as well, by the way.

Of course you question whether or not an observed association has a clear biological or social mechanism. And you attempt to control for other variables that might influence the link between your exposure and your outcome. You run followup studies in different populations to try to understand if the result is a widespread phenomena, or a fleeting bit of statistical noise.

Basically, you do your job well. Which is why I used phrases like "a good first step".

Your example is about as useful as "Programming is useless because once I coded something poorly and corrupted my data".

[ekianjo]

You didn't say proof but you said it's better than guessing, and I don't agree with you at all. What if there is a correlation between Vegetarian-lifestyle and Serial-killers ? Does it tell you that it's better than guessing ? Do you even question if the association/correlation makes remote sense ? Is there any underlying mechanism of action that would remotely explain rationally why this correlation could be linked to any real causation phenomenon ?

Correlation is useless, and there's a ton of observational studies out there finding correlations every single day for which we have no rational explanation at all. Observational studies are full of variations in the way they are designed, the way they are reported and the subjects of the studies, it's rather a miracle if you actually detect a hint of causation based on the garbage noise that you get.

[mattfenwick]

> make a determined effort to control for as many "third factors"

How can you tell if you missed one?

> [...] gives you a level of evidence which is well above guessing

> It is better than guessing

Why? Without any support, this seems to be an appeal to probability.

> a well conducted observational study

> a really bad study

The fallacy of moving the goalposts; also the no true Scotsman fallacy.

[Fomite]

> How can you tell if you missed one?

Similar studies, using as many variables as you can find. Residual confounding is always and ever a problem, but the odds that something is both a strong residual confounder and has never been observed to have an association with the outcome or the exposure is pretty rare?

> Why? Without any support, this seems to be an appeal to probability.

It's really not - if for no other reason than it's forced you to think about your system more than a simple guess would. It's not an appeal to probability, its using data to update whatever prior you came in with. Guesswork is just using your prior.

> The fallacy of moving the goalposts; also the no true Scotsman fallacy.

Not really, no. Some observational studies are crap - this is just true. But that doesn't say anything about the potential quality of observational evidence, and many of the commonly raised objections to observational studies are actually objections to poorly run studies. The example used was a study that examined no potential confounding variables, looked at a correlation with no prior evidence suggesting any linkage between the two or biological plausibility, and then asserts that they've found a causal link.

That's a bad study. It's not 'No True Scotsman Fallacy' to say that the problems with a bad study don't generalize to all studies. If it is, then we're all screwed, because you can run a bad RCT too.

[mattfenwick]

> It's really not - if for no other reason than it's forced you to think about your system more than a simple guess would.

Appeal to probability; appeal to common sense.

> Some observational studies are crap - this is just true.

Special pleading.

> But that doesn't say anything about the potential quality of observational evidence

Straw man.

> many of the commonly raised objections to observational studies are actually objections to poorly run studies

> That's a bad study.

Moving the goalposts. No true Scotsman.

> It's not 'No True Scotsman Fallacy' to say that the problems with a bad study don't generalize to all studies.

Straw man.