Hacker News new | ask | show | jobs
by whoistraitor 544 days ago
Perplexity is listed, but do they actually abide by llms.txt? And how can we prove they do? Is it all good faith? I wish there were a better way.
1 comments

llms.txt isn't an opt-out signal like robots.txt, it's a way to provide a simplified version of pages that are easier for LLMs to ingest. It's more of an opt-in for being scraped more accurately.
Or scraped inaccurately. It seems like you could have some fun with this if you were so inclined...