Hacker News new | ask | show | jobs
by Diti 547 days ago
If one doesn’t want LLMs to scrape data and knows the LLMs will be ignoring the robots.txt file.