|
|
|
|
|
by notafraudster
2935 days ago
|
|
Great example. In the design you propose, in expectation, you would have an unbiased causal inference. We would probably want to check for pre-treatment balance between groups to make sure that stochastic (chance) imbalance did not emerge even though the process itself is good. I don't know anything about heart attacks so I don't have the subject matter knowledge here, but imagine that smoking causes heart attacks. If that's the case, although your design should not cause the presence of smokers among treated and control units to systematically vary, maybe it did by chance. We'd want to assess balance. Same with any other potential confounders. Another technique we might use is a blocked (or stratified) random sample. Knowing that there will be both smokers and non-smokers, we recruit two separate samples, and randomize treatment assignment within each. This ensures that smoking status does not predict treatment assignment and guards against some potential threat from overall randomization. We could also mitigate the imbalance that does exist by doing a matched analysis, where each treated unit is paired with a control unit that looks most like him (some control units are reused). Or we could match on propensity scores. Or we could weight on inverse propensity weights. Or we could weight using covariate balancing. Or... My point in doing this info dump is to a) back up nerdponx's example, which is great and b) illustrate how there's a lot to learn about how statisticians have taken the problem of causal analysis seriously and developed techniques appropriate for answering causal questions. People in the CS side of things tend to use Pearl's DAGS for conceptualizing this stuff. I'm in the stats/econ side of things so I use Neyman-Rubin. They're equivalent. Allow me to suggest Rubin and Imbens - Causal Inference for Statistics, Social and Biomedical Sciences as a good textbook that we assign to graduate students learning this stuff. Some of my students tell me the "Causal Inference Mixtape" is popular among people who want less statistical theory and more "what should I do as a practitioner". A virtue of both the resources I just mentioned is that they discuss not just experimental designs but also observational data studies, like the one the original post would have wanted to conduct. |
|