|
|
|
|
|
by cryptonym
2 days ago
|
|
Not a fair statement. Google wasn't built on bypassing bot protections. Google is providing a service to the websites they crawl. They try to not crawl when we don't want them (robots.txt, clear user-agent, no-index no-follow...). |
|
Yeah they're building an LLM and making it pointless to visit the websites.