Hacker News new | ask | show | jobs
by 6gvONxR4sf7o 2500 days ago
>observational study, which can simulate A/B experiments

This is 100% overselling. Observational studies can be suggestive, but cannot replace experiments. Unobserved variables cannot be accounted for.

1 comments

Thanks for the feedback! I totally agree about observational studies being suggestive but not replacing A/B tests - that’s why the main use case I listed in the blog (and how current customers have used the product so far) is “prioritization of a/b tests”, not replacing a/b tests themselves. The language around “simulating a/b tests” is just a way to try to concisely explain to someone at a high level who may not be very technical or have much experience with causal inference. Happy for suggestions on how to better convey this without over-selling!