Do AI bots actually read llms.txt? 8 days of server logs

llms.txt is a proposed file at the root of a site, like robots.txt, that gives AI models a short Markdown map of the important pages. Lots of guides now tell you to add one. The honest question is whether any AI system actually fetches it. So we checked our own logs.

The setup

• Site: pagelens.io, launched on 7 October 2026, with an llms.txt from day one.
• Every request is logged on the server, so bots that don't run JavaScript are included. User agents are matched to AI companies, and where the company publishes IP ranges, the IP is checked against them.
• Window: 8 to 15 October 2026. One small, new site, so treat this as one data point, not a study.

What the logs show

The six llms.txt requests came from six different bots, once each:

Googlebot fetched it twice as well. Google has said publicly that it doesn't use llms.txt, so that's most likely ordinary crawling of a URL it found.

What this means

• It does get fetched. Every major AI company's bot requested it within a week. So the file isn't invisible.
• It isn't treated like robots.txt. ClaudeBot asked for robots.txt 46 times in the same week and for llms.txt once. Nobody is re-reading it before each crawl.
• We can't tell whether it changed anything. A fetch shows a bot downloaded the file, not that it followed the links or used them in an answer. The pages that got the most AI reads (our study of 740 sites, the home page, the checker) were linked from everywhere anyway.
• ChatGPT-User fetched it once. That agent acts for a person in a conversation, so someone's question led ChatGPT to open our llms.txt. One event, but it's the only sign here of the file being used at answer time.

Should you add one?

Yes, but for the right reason. It takes ten minutes, costs nothing and the bots do download it. Just don't expect it to get you into AI answers. What moves that is the same as before: AI bots allowed in robots.txt and at your CDN, content in the HTML, and pages that other sites link to. Here's how to check the first three.

If you add one, keep it short, link your best pages with one line each, and keep it current. You can check whether a site has one with the free AI crawler checker.

We'll rerun this when the site has a month of data. If you have your own logs, the command is one line: grep -i "llms.txt" access.log | grep -Ei "GPTBot|ClaudeBot|PerplexityBot|OAI-SearchBot|ChatGPT-User".