How to check whether ChatGPT can see your website

Why asking the chatbot does not work

The intuitive test — open ChatGPT and ask about your company — is unreliable. A plausible-sounding answer may come from training data rather than a live fetch of your site, so a good answer does not prove your site is readable, and a bad answer does not tell you which layer failed.

The layers that actually determine citability are testable directly, and each one produces a definitive yes or no.

Step 1 — robots.txt policy

Fetch your robots.txt and check whether the retrieval crawlers are permitted. A blanket Disallow that catches OAI-SearchBot is the most common single cause of invisibility.

curl -s https://yoursite.com/robots.txt

Step 2 — does the request actually complete

Send a request as the crawler and check the status code. A 403, 429 or a challenge page means your CDN is blocking it regardless of what robots.txt permits.

curl -s -o /dev/null -w "%{http_code}\n" \
  -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot" \
  https://yoursite.com

Step 3 — is your content in the HTML

Count the words in the raw response. If it is dramatically lower than what you see in a browser, your content is client-side rendered and AI crawlers are not getting it.

curl -s https://yoursite.com | sed 's/<[^>]*>//g' | wc -w

Or run all three at once

Crawlable runs these checks against fourteen AI crawlers, compares raw HTML against a full JavaScript render, and returns a report with the evidence for each finding. It is free and needs no account.

Frequently asked