robots.txt vs llms.txt for WordPress: what each one does
robots.txt and llms.txt are related, but they are not interchangeable. The first is primarily about crawl policy. The second is primarily a public guidance and context file. A mature WordPress setup can need both.
What robots.txt does
It expresses crawl posture: which paths should or should not be crawled, which user-agent families matter, and how the site wants automated access to behave at the file level.
What llms.txt does
It helps publish a structured machine-readable guide about important content or interpretation. It is better understood as a companion guidance file than as a blocking file.
Why Better Robots.txt pairs them
The plugin can pair both because the real WordPress need is not "another standalone file." The real need is a coherent policy workflow for search, AI, and machine-readable guidance.
What Better Robots.txt is not
Better Robots.txt is not a WAF, not a signed-agent verification system, not a legal enforcement layer, and not a guarantee that every crawler will comply. It publishes a clearer WordPress policy surface and a safer workflow for the parts you can actually govern.