Y
Hacker News
new
|
ask
|
show
|
jobs
by
jsheard
166 days ago
If synthetic data is so much better then what are AI crawlers still DDOSing everyone for? Are they stupid?
1 comments
philipkglass
166 days ago
Mostly. I had the "AI bot tsunami" problem on my own personal site and blocked a bunch of bot user agents in robots.txt. Most of them were from companies I had never heard of before. The only big AI name I recognized was GPTBot from OpenAI.
link