GPTBot, OAI-SearchBot, ClaudeBot: who is knocking
Your server writes down a visitor called GPTBot. Later it writes down OAI-SearchBot, then ClaudeBot, then a few more. They come from a handful of companies, and they are not doing the same job. Telling one of them to stay away tells the others nothing. Here is which is which, taken from the pages those companies publish themselves.
1. Three jobs, not nine problems
Nine names sounds like a lot to hold. It is really three jobs.
Training readers collect pages that may be used to build the models themselves. GPTBot from OpenAI and ClaudeBot from Anthropic sit here.
Search readers gather pages so an assistant can show and link them when someone asks a question. OAI-SearchBot, Claude-SearchBot and PerplexityBot sit here.
Errand readers fetch one page, once, because a person has just asked for it. ChatGPT-User, Claude-User and Perplexity-User sit here.
The ninth name, Google-Extended, is not a visitor at all. More on that below, because it catches people out.
2. The training readers, and what a block actually says
OpenAI describes GPTBot as crawling content that may be used to train its models, and says that disallowing GPTBot indicates your content should not be used in that training.
Anthropic describes ClaudeBot as collecting web content that could contribute to training its models.
That is the whole of it. Blocking a training reader is a statement about training. It is a reasonable thing to want, and plenty of shop owners want it.
Here is the part worth slowing down for. OpenAI states that the search and training controls are independent. It gives allowing OAI-SearchBot while blocking GPTBot as an example. When both uses are allowed, OpenAI may reuse one crawl for both purposes.
3. The search readers, and what a block actually costs
OAI-SearchBot is the one OpenAI says is used to surface websites in ChatGPT's search features. The consequence is stated plainly on the same page: sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though OpenAI notes they can still turn up as plain links.
PerplexityBot is described as surfacing and linking sites in Perplexity's results, and Perplexity goes one step further than the others. Its page says the bot is not used to collect content for training its models. That is the clearest statement any of these companies makes that the two jobs really are separate.
Claude-SearchBot is Anthropic's equivalent, described as navigating the web to improve search result quality.
So if being findable inside an assistant matters to your shop, these three names are the ones that carry it, and they are the ones a blanket block quietly takes away.
4. The errand readers, and why your file may not stop them
ChatGPT-User, Claude-User and Perplexity-User are different in kind. They turn up because somebody asked a question a moment ago and the assistant went to look.
OpenAI writes that because these actions are started by a user, the rules in your file may not apply. Perplexity is blunter still and says that since a user requested the fetch, its fetcher generally ignores those rules. Anthropic presents its three readers as following the file.
Two things follow, and they pull in opposite directions.
Your rules file is a request, not a lock. Two of the companies have written down that they may walk past it in this case. That is not sinister, it is what a person clicking a link looks like from the other side, but you should know it before you assume a line in a file settled anything.
And letting the errand readers in is not a route into search answers. OpenAI says ChatGPT-User is not used to decide whether content may appear in Search, and that it does not crawl the web automatically. It fetches what it was sent to fetch.
5. Google-Extended never knocks
This one confuses almost everybody, so it is worth being exact.
Google-Extended will never appear in your server records, because it has no visitor name of its own. Google says it does not have a separate request name at all. The crawling is done by Google's ordinary visitors, and Google-Extended is only a word you can put in your file to control what happens to what they took.
What it controls is training and grounding for Google's Gemini models. What it does not do is also stated plainly: Google says Google-Extended does not affect a site's inclusion in Google Search, and is not used as a ranking signal there.
There is also no word you can put in the file for AI Overviews or AI Mode. Google's position is that AI is built into Search, so the ordinary rules for Googlebot are the control. Google has since added a separate setting in Search Console, under Settings and then Search generative AI, and says that control is not used as a ranking or inclusion signal elsewhere in Search.
6. Anyone can type a name
Developer needed, or at least someone who reads server records for a living.
A visitor name is just text the visitor writes about itself. Anything can claim to be GPTBot. If you are about to build rules around these names, the names alone are not enough.
Each company publishes the network addresses its readers really use. OpenAI publishes a separate list for each of its readers, and it has added readers since this was written. Perplexity publishes one for each of its two. Anthropic publishes a single combined list covering all three of its readers, which means you can confirm that a visitor is Anthropic's but not which of the three it was.
Anthropic adds a warning worth repeating. Blocking by network address is a poor way to say no, because a reader that cannot reach your site cannot read your rules file either, so it never learns what you wanted.
7. What to do with your own file
You can do this. Open yourstore.com/robots.txt and read it name by name.
Then make three decisions instead of one. Do you mind your writing being used for training? Do you want your shop findable inside these assistants? And are you comfortable that a page someone links to may be fetched regardless?
Those answers can differ, and the file is perfectly able to hold three different answers at once. Our access check tests the site-root rules for nine tracked names and control tokens, then lists any it finds blocked or partly restricted. It does not prove that a visitor reached the site.
What this does not do
Reading the file tells you what you asked for. It does not tell you what happened. Nothing here confirms that any assistant has actually read your shop, and two of these readers are documented as possibly ignoring the request anyway.
Your protection service can also turn any of them away long before your file is consulted, which is a separate problem with a separate playbook.
And this list is a snapshot. These companies add names, rename them and change what they say about them. Everything above is what those pages said on 25 August 2026, and it is worth rereading once or twice a year.