Rankavex

Check 01 / 05

Does blocking GPTBot in robots.txt stop ChatGPT from citing your site?

No, not by itself. GPTBot only collects pages to train OpenAI’s models; ChatGPT search uses a different crawler, OAI-SearchBot, and OpenAI documents the two as independent. What stops ChatGPT from citing you is a robots.txt rule that blocks OAI-SearchBot — by name, or through a “User-agent: *” group with “Disallow: /”.

# Takes a site out of ChatGPT search
User-agent: OAI-SearchBot
Disallow: /

# Only opts it out of OpenAI’s training
User-agent: GPTBot
Disallow: /

Two rules that look alike. Only the first takes a site out of ChatGPT search; OpenAI documents the two crawlers as independent.

It’s easy to treat GPTBot as the switch for ChatGPT, and the same shortcut fails at the other assistants. At Anthropic, ClaudeBot is for training, while Claude-SearchBot and Claude-User serve answers. Perplexity is the exception that proves the rule: PerplexityBot is its search crawler, so blocking it does remove you from Perplexity’s answers.

To check your own site, open yourdomain.com/robots.txt and sort what you find into two piles. Rules that decide whether assistants can cite you: OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User, PerplexityBot and Perplexity-User — plus any “User-agent: *” group with “Disallow: /”, which blocks all of them. ChatGPT-User and Perplexity-User may ignore the rule anyway, by their operators’ own documentation. Rules that only opt you out of training: GPTBot, ClaudeBot, CCBot, Applebot-Extended and Google-Extended — read that last one twice, because it also opts you out of Gemini grounding. The AI crawler directory covers all 25 user-agents, with the source for each.

A rule in the first pile is worth questioning: it often comes from a catch-all block or an old anti-scraper template nobody revisited. A rule in the second pile is usually a deliberate choice, and a defensible one — removing it won’t make you more citable. Being reachable by the right crawlers is necessary, not sufficient — see the other checks below.

Want to know what an AI sees when it reads your site?

Send the domain. You'll get the findings back either way.

Ask for a free look at your site

Sources

Cite this

No, not by itself. GPTBot only collects pages to train OpenAI’s models; ChatGPT search uses a different crawler, OAI-SearchBot. What stops ChatGPT from citing you is a rule that blocks OAI-SearchBot, by name or through “User-agent: *”.

Igor Almeida, Rankavex. Last updated 2026-09-21
https://rankavex.com/checks/robots-txt-blocking-gptbot/

All five checks, in order →