Hacker News new | ask | show | jobs
by aguyfromnb 2234 days ago
>Take an audience of X people. Divide them in two. Show ads to your test group, don't show to control. Watch your business grow and gauge the lift between the two audiences.

How are you measuring the group you don't show ads to?

This is exactly the sort of experiment that the economists in this article (both independent and from Facebook) did, and measured substantial selection effects.

The fact that there are businesses built on businesses built on businesses that are, essentially, allowing people to compete for tiny ad space on specific keywords demonstrates that there is too much hype and obscurity in the space.

1 comments

In this case, you create a test and control of two audiences using a variety of techniques:

1. Do 1st party -> 3rd party data matching with a partner like LiveRamp or Oracle (formerly DataLogix), where you effectively take consumer data you have gathered and then match it against a targeting set. Then you select one group as test and one as control. Depending on the size and characteristics of your dataset, you often will work to randomize that audience across different traits (age, gender, HHI, etc) to ensure that you get two groups that are effectively the same. Your primary purpose here is retargeting/upsales or using data you have purchased to drive new customer acquisition.

2. Do a 1st party data -> 3rd party lookalike with a partner like Facebook or Google, which is then randomized into two groups for test and control through their own system. Pretty straightforward and great for acquisition.

Then you run ads to your test group and none to control. Or maybe control sees your current message, and you test a new one with the test group. Or maybe you have more than one test group to gauge the lift on your business. Lots of options.

Now, you gauge your results by looking at the fundamental shift in consumer behavior that occurs in your test group vs. control. You're really trying to say "hey, we showed X ads at $Y cost to this group of Z people. Our control group saw 2% become customers, but the test group saw 6% become customers. Thus our incremental lift is the amount over the baseline of control."

This is a totally oversimplified way of doing it because the reality is vastly more complex, but that is the gist of it. Many advertisers and agencies are savvy to this methodology, but an even larger number are not. Those that test and iterate this way are the ones who are effective at driving growth.

Now, to your second point:

> The fact that there are businesses built on businesses built on businesses that are, essentially, allowing people to compete for tiny ad space on specific keywords demonstrates that there is too much hype and obscurity in the space.

Don't conflate bottom of funnel high-intent search ads with your general digital ads. In the case of Google, you are bidding on intent at that moment, probably one of the most valuable things you can advertise against. This is why Google prints so much money - that search, such as "used 2020 MacBook Pro for sale" is about the highest indicator of someone who intends to buy you can get outside of them banging on your door saying "take my money!!"

This is about the most incremental advertising you can ever get for 99% of companies that are not Amazon. Amazon doesn't care because most of the time people go search on Amazon first, but they're still willing to advertise for the 0.001% of their customers who aren't starting with them.