[Semaphor]

On the other hand, they are amazing at catching bots. Almost all bots (obviously excluding disguised ones, but those never were an issue for us) have identifiable user agents, by blocking bots via UAs, we became better than Google Ads at blocking bots, they are probably doing some kind of complicated ML thing that works far better for edge cases, our simple solution works better for normal cases…

[secondcoming]

Even today? UA strings are easily fakeable (so fakeable that it surprises me that people still use them for anything).

If a bot still gets caught by UA strings then it's just a poorly written bot?

DoubleVerify is a Googley company that does bot detection. That uses the UA and IP address to find them.

[Semaphor]

I’m talking about actual, legit bots. Facebook, Instagram, all those search engine crawlers. Those follow all kinds of links, including ads, and then go and annoy us and the advertisers by counting as "fake traffic".

Google is/was (we wrote our own simplified adserver, only using AdManager for the agencies that require it, so I’m not sure how much things changed in the last two years) not only happy letting those through, they even send their own, it was so bad that we redirected all links through our site where we filtered all Google IP ranges (because, of course, whatever they used did not have a proper bot UA) that we could find to block them and stop sending 1000s of fake visits to the advertiser every day.

[Multicomp]

I wonder if these bots would respect robots.txt files for the ads?

[Semaphor]

Well, you’d have to get Google to host those robots.txt as the ads are running iframed on their servers ;)

[Avamander]

Easily fakeable and abusers still use bad ones. The bar is barely above the floor.