Skip to main content

Free tool · No signup

Is your website blocking AI search from reading it?

When someone asks ChatGPT or Perplexity for the best business like yours, it can only cite your pages if its search crawler is allowed to read them. Some sites refuse AI crawlers at the server while their robots.txt says everything is fine. The standard SEO check misses it. This one doesn’t: we fetch your site as each engine’s search crawler and show you what comes back.

We fetch your homepage as each AI engine’s search crawler and compare it to a normal browser. No signup. Takes ~15 seconds.

A buyer asks ChatGPT

“Which firms should I consider for my business?”

The assistant cites the competitors whose pages it could fetch and read, and never cites your site, because its search crawler never got in.

If your server quietly blocks an engine’s search crawler, your own pages can’t be its source. It may still name you from what other sites say, but it can’t quote you, and nothing tells you. The check above shows what each crawler actually sees.

Checked acrossChatGPT · Claude · Perplexity · Gemini

From Edition 2 (October 2026) the SA AI Visibility Index reports this same search-crawler test, beside the three-crawler test Edition 1 ran. It’s our public study of how AI assistants answer for South African brands. See exactly how we test →

Why this matters

This isn’t our inference. OpenAI states it plainly: sites opted out of OAI-SearchBot “will not be shown in ChatGPT search answers” (OpenAI’s crawler documentation).

Crawl permission is the first of two conditions OpenAI’s ChatGPT search help page puts on being eligible: allow OAI-SearchBot, and let its published IP ranges through your host or CDN. Eligible is not cited (OpenAI says placement is not guaranteed), but an opted-out site is left out of ChatGPT’s search answers, and both gates live at the server, where an infrastructure rule nobody in marketing ever sees can flip them.

GPTBot is a different crawler: OpenAI uses it for model training, and blocking it doesn’t take you out of ChatGPT search.

Being readable makes your own pages citable. It doesn’t get you recommended, and engines regularly name brands from what other sites say about them, citing a directory, a news story or a review site rather than the brand’s own pages. Crawl access is one of seven patterns we measured across 600+ real AI answers about South African businesses in property, banking and crypto (the study). The others: each engine answers differently, they lean on sources they already trust, they prefer pages that answer a question over pages that list your address, and schema markup matters far less than most GEO advice claims. Crawl access is the cheapest one to fix, usually a one-line change once you know to look.

See where your brand actually stands across all four engines →