|
|
|
|
|
|
by jjangkke
347 days ago
|
|
|
so TLDR it adjusts your robot.txt and relies on cloudflare to catch bot behavior and it doesn't actually do any sophisticated residential proxy filtering or common bypass methods that works on cloudflare turnstill, do I have this correct? this just pushes AI agents "underground" to adopt the behavior of a full blown stealth focused scraper which makes it harder to detect. |
|
|