The AI search readiness checklist

A practical checklist to help AI engines read, trust and cite your site: crawler access, facts as text, llms.txt, schema and buyer questions.

· 6 min read · By the Massyl team

AI engines can only name your brand if they can read your site, understand what you sell, and trust that your facts are right. Most of that comes down to a few practical checks.

This checklist covers the basics. None of it guarantees a mention. AI answers vary, and no one controls them. But each item removes a reason for an engine to skip you or get you wrong.

1. Let AI crawlers into your site

Your robots.txt file tells bots what they may read. Many sites block AI bots by accident, through a blanket rule or a security plugin. Check yours first.

It helps to know that AI bots fall into two broad groups:

  • Search and fetch bots read your pages so an engine can answer a question right now and cite you. Blocking them can keep you out of live answers.
  • Training bots collect pages to train future models. Blocking them does not usually remove you from live search answers, but your content will not shape what the model learns.

You can allow one group and block the other. Here are the main bots to know:

Bot (user agent) Company What it is for
OAI-SearchBot OpenAI Finds pages to show and cite in ChatGPT search
ChatGPT-User OpenAI Fetches a page when a ChatGPT user’s request needs it
GPTBot OpenAI Collects pages for training OpenAI models
PerplexityBot Perplexity Indexes pages for Perplexity answers
Claude-SearchBot Anthropic Indexes pages to improve Claude’s search results
ClaudeBot Anthropic Collects pages for training Claude models
Googlebot Google Crawls for Google Search, including AI Overviews and AI Mode
Google-Extended Google A control token for Gemini training and grounding, not Google Search
Bingbot Microsoft Crawls for Bing, whose index also feeds Microsoft’s AI answers
Applebot-Extended Apple A control token for using pages to train Apple’s AI models
CCBot Common Crawl Builds an open web archive widely used to train models
Meta-ExternalAgent Meta Collects pages for Meta’s AI products and training

Two points people often get wrong:

  • Google-Extended does not affect Google Search. Blocking it does not remove you from Google results, AI Overviews or AI Mode, which use Googlebot. It controls whether your content is used for Gemini training and grounding.
  • Blocking Googlebot to keep out of AI Overviews also removes you from Google Search. There is no separate AI Overviews bot to block.

Check it now: paste your robots.txt into our free AI crawler checker. It shows which AI bots are allowed and which are blocked. It runs in your browser and sends nothing anywhere.

Also check your firewall, CDN or bot protection settings. A robots.txt that allows a bot does not help if your host blocks it before it reaches the page.

2. Put key facts in text, not images

Engines read text. If your opening hours sit inside a photo of your shop window, or your prices are only in a PDF menu scan, an engine may miss them or guess.

Make sure these appear as plain HTML text on a page that is easy to find:

  • What you sell and who it is for
  • Prices or price ranges
  • Opening hours and locations
  • Shipping, returns and service areas
  • Contact details

Wrong facts in AI answers often trace back to facts that were hard to read or that changed without the old version being removed.

3. Add an llms.txt file

llms.txt is a proposed convention: a plain text file at the root of your site that gives AI tools a short summary of who you are and links to your most important pages. Think of it as a guided tour for a reader that is short on time.

Be realistic about it. It is a proposal, not a standard, and adoption across AI engines varies. It is cheap to add and does no harm, but do not expect it to replace the other items on this list.

Make one: the free llms.txt generator builds a draft from a few details you type.

4. Add schema (JSON-LD)

Schema markup is structured data in your page code that states facts in a format machines read easily: your business name, address, hours, products, prices, reviews and FAQs. It is written as JSON-LD inside a script tag.

Useful types for most businesses:

  • Organization or LocalBusiness for who you are
  • Product and Offer for what you sell and the price
  • FAQPage for common questions and answers

Keep schema in sync with the visible page. If the schema says you open at 7am and the page says 8am, you are giving engines two answers to choose from.

Make it: the free schema generator produces JSON-LD you can paste into your site.

5. Publish honest comparison pages

Buyers ask AI engines things like “X vs Y” or “best alternatives to X.” If your site has no page that answers that, the engine will lean on someone else’s page, often a competitor’s or a review site’s.

A good comparison page:

  • Names the alternatives plainly
  • Says who each option suits, including when a competitor is the better fit
  • Uses facts that can be checked: prices, features, locations
  • Is kept up to date

Before writing a new page, check whether an existing page can be improved to cover the question. Fewer, stronger pages are easier to keep accurate.

6. Answer your buyers’ questions on your site

AI engines answer questions. The more directly your site answers the questions your buyers ask, the easier it is for an engine to use you as a source.

Start by listing those questions. Think about what a real buyer types, not what you want to rank for. For a made-up example business, Halden Coffee, a fictional roaster in Austin, that might be:

  • “Where can I buy fresh roasted coffee beans in Austin?”
  • “Which Austin roasters offer a coffee subscription?”
  • “Who supplies wholesale coffee to cafes in Austin?”

Then make sure each one has a clear answer somewhere on your site, ideally in the first lines of a relevant page.

Get a list: the free buyer question generator drafts questions from your business type.

7. Keep your facts consistent everywhere

Engines do not read only your site. They also read listings, directories, review sites, social profiles and marketplaces. If your hours are different on your site, your Google Business Profile and a directory, the engine has to pick one, and it may pick the wrong one.

Go through the places your business is listed and make the core facts match:

Fact Where to check
Name and address Your site, map listings, directories
Hours Your site, map listings, social profiles
Prices Your site, marketplaces, menus
What you offer Your site, review sites, social bios

Remove or update old pages that state outdated facts. Old pages can keep feeding engines the wrong answer long after you have moved on.

Get a quick score

Want a starting point? The free AI readiness score walks you through these checks and gives you a score with the items to fix first. Like all our free tools, it runs in your browser and sends nothing anywhere.

Then measure

A checklist tells you what to fix. It does not tell you whether engines now name you more often. For that you need to ask the engines your buyers’ questions many times and track the rate over time. That is what Massyl is being built to do, with ranges on every number and proof of which changes worked.

If that sounds useful, join early access or read about the founding prices. For how the measuring works, see how Massyl measures.

Get your free 20-run report.

Twenty answers per question, every answer, every range. Free when Massyl opens, no card.

Free report when Massyl opens. One email, no sales calls.