[white_viel]

Finally managed to kill the traffic: 1. renamed the worker - the bot was bypassing the route on the domain and using the worker.dev domain 2. add WAF rules to block gptbot in user agent 3. serve zip bomb on request from GPT.

interesting that GPT was accessing the worker directly to bypass the WAF rule son Cloudflare

[kentonv]

Interesting. workers.dev domains can be a liability sometimes -- if you've mapped the worker to a real zone, then you probably don't want the workers.dev zone anymore.

For what it's worth, you can disable the workers.dev zone by putting `"workers_dev": false,` in wranlger.jsonc. You can also enable Cloudflare Access on your workers.dev zone to require login (there's a switch for this in the cloudflare dashboard UI for the worker).

But of course you have to remember to do those things... I wonder if we (Cloudflare) should be more proactive in suggesting disabling/locking down the workers.dev zone once a worker is mapped to another zone...

[white_viel]

>> interesting. workers.dev domains can be a liability sometimes -- if you've mapped the worker to a real zone, then you probably don't want the workers.dev zone anymore.

maybe that is a good idea, prompt a user to disable the worker.dev domain. however, having the worker.dev domain open for me is more of a backup way to access the worker as the mapped domain is more for a hobby project.

> Interesting. workers.dev domains can be a liability sometimes

what about allowing the specific worker.dev to serve a robots.txt on its own that can be used to disable the AI bots?

i.e https://XXXXXXXXXXX.username.workers.dev/robots.txt should be allowed to be configure on a worker level. not sure how that affects the design of the infra, but it would be a good idea

[white_viel]

but i am impressed that gpt was able to decipher the worke.dev url on its own and access it.