Hacker News new | ask | show | jobs
by reconnecting 84 days ago
Not at all, only web crawlers from hetzner network.

https://news.ycombinator.com/item?id=46058273

1 comments

I have seen something similar in a site I work on
There (1) is more context on this topic, where author of LLMs.txt idea said 'llms.txt files have nothing to do with crawlers or big LLM companies. They are for individual client agents to use.'

1. https://news.ycombinator.com/item?id=47058870