[HermanMartinus]

I can definitively say llms.txt is not used by any AI players. I run a blogging platform with around 80k blogs and /llms.txt is not requested by anything (other than humans checking to see if there's an llms.txt path).

All regular pages are aggressively scraped to the extent it's a problem I have to consistently manage, but not llms.txt.

[sunshine-o]

Amazing, I didn't know.

So it get even stranger, I am the only one reading those /llms.txt ...

[nickserv]

I'm seeing quite a bit of request for these on my work's GitBook documentation site.

But perhaps these are developers specifically targeting these pages to feed whatever LLM they are using.

[isaachinman]

How is a static blog being scraped a problem? Do you not use a CDN?

[nickserv]

> a blogging platform with around 80k blogs

But nah, I'm sure OP doesn't know about CDNs.

[the_real_cher]

Are all blogs static though?

[johannes1234321]

Very few blogs require frequent updates. Even with user comments.

[0123456789ABCDE]

> I can definitively say llms.txt is not used by any AI players.

  https://developers.openai.com/llms.txt
  https://docs.anthropic.com/llms.txt
  https://geminicli.com/llms.txt
  https://github.com/llms.txt
  https://docs.aws.amazon.com/llms.txt
  https://openrouter.ai/docs/llms.txt

[m4tthumphrey]

OP clearly meant that the AI players are not reading and/or honouring llms.txt of other websites when scraping.

[0123456789ABCDE]

i stand corrected, but what was clear to you, obviously was not clear to me.