Study: 97% of llms.txt Files Never Read by AI Crawlers
The week's most data-backed story came from Ahrefs, which analyzed server logs and bot traffic across 137,000 domains and found that 97% of llms. txt files received zero requests from AI crawlers. On top of that, AI retrieval bots accounted for just 1% of total crawl traffic across all sites studied. The findings, covered by both Ahrefs directly and Search Engine Journal, represent the most concrete evidence yet that the llms.
Read issue