Anthropic’s Claude Bots Make Robots.txt Decisions More Granular via @sejournal, @MattGSouthern
Last updated Feb 26, 2026 by ATDb automated enrichment
Details
Anthropic updated its crawler documentation to list separate Claude bots for training, search indexing, and user requests, with visibility tradeoffs when blocked. The post Anthropic’s Claude Bots Make Robots.txt Decisions More Granular appeared first on Search Engine Journal.