Hacker News new | ask | show | jobs
by aditiyaa1 2427 days ago
Hi Bilal, Thanks for the overview of the product. This is a really important business problem to solve for many marketing teams. Just using this to prioritize A/B tests in itself pretty valuable. But one of the concerns around this approach is the un-reliability of causal analysis to estimate true effects. The link below refers to a study done at FB that shows observational studies could be erroneous in estimating effect sizes and in some cases, the direction of the effects. Do you think clearbrain's system is robust enough to estimate the true effects?

https://www.kellogg.northwestern.edu/faculty/gordon_b/files/...

1 comments

Thanks for the great feedback! Yes, some of these limitations expressed in the study are true in the case of ClearBrain - namely we are leveraging observational studies at this time as a prioritized ranking algorithm for which behaviors are most important, but the actual effect sizes themselves may be variable. We're working on improvements, as well as incorporating actual experiment data into our algorithm to make it more accurate over time.
Thanks! What is your strategy around incorporating actual experiment data? Not sure I fully follow here.