When you opt in, Cloudflare will automatically deploy an AI-generated set of linked pages when we detect inappropriate bot activity, without the need for customers to create any custom rules.
AI Labyrinth is available on an opt-in basis to all customers, including the Free plan.
It's opt-in for now anyway so if it is causing pain people should find a way to contact the website operators in question and have them open tickets assuming they are not on the free plan and get the AI tuned. When all else fails they can create Tell HN threads here and provide details. Sometimes those threads get the attention of Cloudflare executives here. I would bookmark these [1][2]. Excluding non-executives that are also here.
I am personally not against the idea of having squirrel wheel traps for bots as I have created very simplistic ones in the past that worked well against poorly coded bots and sometimes even crashed them to the point where bot operators would block my domains from being crawled. I do not have the skills of CF to make something more advanced like they did or I would and since I do not use CDN's I am on my own unless someone makes an open source version that can be plumbed into HAProxy or Nginx. I guess that makes me a skiddie.
"Bot attack protection" mode is also opt-in and many sysadmins just enable it immediately after setting up their website with CF and forget about it (because Cloudflare's marketing suggests it), leading to legitimate browsers being blocked: https://news.ycombinator.com/item?id=43329320
With so much hate towards LLMs right now (which isn't unjustified) being vented on the internet there's no doubt sysadmins will do the same here and niche user agents will again suffer.
> AI Labyrinth is available on an opt-in basis to all customers, including the Free plan.
I do not see it in the free plan. Per the screenshot in the article, on the bots section I see two toggles - Bot Fight Mode and Block Bots. Below these toggles I see
1. A call to action Upgrade Plan for a Super Bot Fight Mode (pro or business)
> so if it is causing pain people should find a way to contact the website operators in question
Yeah, no. That's silly and no normie knows how to contact website operators, or are likely to even understand they should. Also how would they find the contact of they can't access the website. This is exactly the same situation as their captcha giving you an infinite loop.
Sure, I mean it wont be the case for all websites but there have been times this site was behind Amazons CDN and I know how to contact dang. It will vary from website to website. Some companies hide behind their CDN which is a red flag telling me to avoid that site. If they are a reputable company there will be a way to reach someone. People here often bring up Google as an example of not being able to reach a human and I refer back to my example of a red flag. If a website is not excluding their "Contact Us" from the AI trap that is either incompetence or malfeasance.
I am also concerned about this too as a screen reader user. How do they hide these links from screen readers and assistive technology in general? Did they even test this with AT before deploying it?
... but they defend my cooking blog from the botz!111 (I guess one could also substitute "from AI scrapers" but I guess that ship has sailed)
In the spirit of not pitchforking, it does make it sound like they put some non-trivial energy into making the injections hidden, but I'm with you that monkeying with responses is the road to ruin
AI Labyrinth is available on an opt-in basis to all customers, including the Free plan.
It's opt-in for now anyway so if it is causing pain people should find a way to contact the website operators in question and have them open tickets assuming they are not on the free plan and get the AI tuned. When all else fails they can create Tell HN threads here and provide details. Sometimes those threads get the attention of Cloudflare executives here. I would bookmark these [1][2]. Excluding non-executives that are also here.
I am personally not against the idea of having squirrel wheel traps for bots as I have created very simplistic ones in the past that worked well against poorly coded bots and sometimes even crashed them to the point where bot operators would block my domains from being crawled. I do not have the skills of CF to make something more advanced like they did or I would and since I do not use CDN's I am on my own unless someone makes an open source version that can be plumbed into HAProxy or Nginx. I guess that makes me a skiddie.
[1] - https://news.ycombinator.com/user?id=jgrahamc CTO of Cloudflare
[2] - https://news.ycombinator.com/user?id=eastdakota CEO of Cloudflare