Free tool · runs in your browser

Is your robots.txt blocking AI search?

Paste your robots.txt and see, bot by bot, which AI crawlers can read your site, and which ones decide whether you show up in ChatGPT, Perplexity, Claude and Google AI answers.

Tip: open yourbrand.com/robots.txt, copy everything, paste it above.

Result

OAI-SearchBot

OpenAI · ChatGPT search results

…
ChatGPT-User

OpenAI · Pages a ChatGPT user asks it to open

…
GPTBot

OpenAI · OpenAI model training

…
PerplexityBot

Perplexity · Perplexity search results

…
Perplexity-User

Perplexity · Pages a Perplexity user asks it to open

…
Claude-SearchBot

Anthropic · Claude search results

…
Claude-User

Anthropic · Pages a Claude user asks it to open

…
ClaudeBot

Anthropic · Anthropic model training

…
Googlebot

Google · Google Search, AI Overviews and AI Mode

…
Google-Extended

Google · Gemini training and grounding (not Google Search)

…
Bingbot

Microsoft · Bing and Copilot

…
Applebot-Extended

Apple · Apple model training

…
CCBot

Common Crawl · Open web crawl used by many models

…
Meta-ExternalAgent

Meta · Meta AI

…

Nothing you type is sent anywhere. The tool runs entirely in your browser.

Search bots and training bots are not the same

AI companies run several crawlers. Search bots (OAI-SearchBot, PerplexityBot, Claude-SearchBot, Googlebot, Bingbot) fetch pages so the engine can cite them in answers. User bots (ChatGPT-User, Claude-User, Perplexity-User) open a page when someone asks the assistant to. Training bots (GPTBot, ClaudeBot, Google-Extended, CCBot, Applebot-Extended) collect data to train models.

If you want to be named in AI answers, allow the search and user bots. Whether you allow training bots is a separate choice: blocking them does not remove you from AI search results.

How to read the result

  • Allowed: the bot may read the path you tested.
  • Blocked: a Disallow rule matches. The table shows which group decided it.
  • No rule for a bot means it follows the User-agent: * group, or is allowed if there is none.

Next steps: check your facts are written as text with the AI readiness score, then read the AI search readiness checklist.

Questions about this tool

Does blocking GPTBot remove me from ChatGPT answers?

Not by itself. GPTBot collects data for OpenAI model training. ChatGPT search uses OAI-SearchBot, and pages a user asks ChatGPT to open use ChatGPT-User. You can block training and still allow search.

Does Google-Extended affect Google Search or AI Overviews?

No. Google-Extended controls whether Google may use your content for Gemini training and grounding. Google Search, AI Overviews and AI Mode use Googlebot.

Is this an official check?

It reads your robots.txt the way the robots standard (RFC 9309) describes: the most specific group wins, then the longest matching rule, and Allow wins a tie. Some bots may not respect robots.txt.

Where is my robots.txt?

At the root of your domain, for example yourbrand.com/robots.txt. Open it in your browser and paste its content here.

Measure it for real, free.

The tools tell you what to fix. Massyl tells you whether it worked: twenty answers per question on every engine, with a range on every number.

Free report when Massyl opens. One email, no sales calls.