Skip to main contentSkip to content

Search vs answer vs training permissions on WordPress

A WordPress team may want classic search discovery, selective answer-surface visibility, and a more restricted training posture. Those are three different questions, and the cleanest policy model starts by keeping them separate.

Search is not the whole machine landscape

Classic search crawlers still matter, but they are not the whole picture. Answer surfaces, archives, signed-agent fetches, and training-related agents introduce new distinctions.

What Better Robots.txt can express

The plugin can express part of that distinction through crawler-family separation, AI usage signals, and optional llms.txt, while still keeping the site readable for the team.

What requires something stronger

When the issue becomes signed access, runtime identity, or hard technical blocking, stronger layers than a publication plugin are still required.

What Better Robots.txt is not

Better Robots.txt is not a WAF, not a signed-agent verification system, not a legal enforcement layer, and not a guarantee that every crawler will comply. It publishes a clearer WordPress policy surface and a safer workflow for the parts you can actually govern.