Hacker News new | ask | show | jobs
by kamran210 14 days ago
Exactly, and even without sitemap ai can crawl the webpage content for internal links.

Not to mention the environmental impact of creating extra files just for AI to crawl and eventually use for training