[VBprogrammer]

A phenomenon which reminds me of the canonical survivorship bias story. In WWII they conducted studies to determine where the bullet holes where on aircraft which returned from bombing sorties, in order to determine which parts of the aircraft required armour. It took a statistician to point out that they actual needed to armour those places where they rarely saw damage on returning aircraft, as those parts are most likely the parts where being hit caused the aircraft to not return at all.

Sometimes it requires a bit of a leap of imagination in order to resolve these things.

[tomrod]

Causal inference has made a lot of improvements since WWII, and "if" the advertising company knows what they are doing they run effective A/B or MAB testing; that said, statistical power is typically low because of insufficient sample size for individual companies.

You could pool all ads together, but since each advertising company is independent you get into all kinds of weird path dependencies.

While I wouldn't claim to be an adtech practitioner, I did at one point help a few F500 work through conceptual models of multitouch attribution and other statistical issues. These are very nontrivial issues -- proving advertising effectiveness is very difficult!