The generator doesn't crawl and index your entire website. Instead, for llms.txt, it:
- analyzes your homepage to extract site information
- identifies and categorizes main navigation links
- creates a structured llms.txt file following the specification
This aligns with the llms.txt proposal.
For llms-full.txt, it fetches the content from the links on the homepage. It's a much more complete documentation of your site.
- analyzes your homepage to extract site information - identifies and categorizes main navigation links - creates a structured llms.txt file following the specification
This aligns with the llms.txt proposal.
For llms-full.txt, it fetches the content from the links on the homepage. It's a much more complete documentation of your site.
You can read more about it here: https://llms-txt.io/blog/what-are-llms-txt-files
Or see examples here: https://llms-txt.io/blog/companies-using-llms-txt-examples