[wtf242]

I had the same issue with TikTok/ByteDance. They were using almost 100gb of my traffic per month.

I now block all ai crawlers at the cloudflare WAF level. On Monday I noticed a HUGE spike in traffic and my site was not handling it well. After a lot of troubleshooting and log parsing, I was getting millions of requests from China that were getting past cloudflare's bot protection.

I ended up having to force a CF managed challenge for the entire country of China to get my site back in a normal working state.

In the past 24 hours CF has blocked 1.66M bot requests. Good luck running a site without using CloudFlare or something similar.

AI crawlers are just out of control