Crawlability for AI
Crawlability for AI is whether AI bots in particular can reach and read your pages, which can differ from what Googlebot sees.
Crawlability for AI is whether AI bots in particular can reach your pages and read what's on them. A site can be easy for Google to crawl and still half invisible to AI platforms, because their bots work differently.
If you've done the basics of crawlability for search, most of it carries over. These are the places where AI bots catch sites out.
Where AI bots differ
Scripts are the big one. Google's crawler runs the scripts on a page. Many AI crawlers read only the HTML your server sends. Text that a script adds after the page opens, like prices, reviews or product details, can be missing for them.
There are more bots, with separate jobs. Each AI company may run a training bot, a search bot and a bot that fetches pages for users. Your robots.txt may treat them differently without meaning to, especially if it was written before they existed.
Firewalls and bot protection can block AI bots on their own. A security setting that stops "unknown bots" may stop an AI search bot while letting Googlebot through.
An example
Fernhill's product pages load their prices and tasting notes with a script. In a browser, everything is there. Googlebot runs the script and sees it too. An AI search bot reads the HTML and finds a product name, a photo and nothing else. Answers about Fernhill's coffee quote prices from another shop's site instead, two years out of date.
Moving the prices and notes into the HTML the server sends fixes it for every bot at once.
How to check
Open the page with scripts turned off and see what's left. Check your robots.txt for each AI bot by name. Ask whoever runs your hosting whether bot protection treats AI bots differently.
In AnswerOrbit
The Pages tab in Website shows each page side by side, What a person sees and What a bot reads, and marks blocks that bots miss as Hidden from bots, with how to fix them. The Page tester checks whether a single page is open to bots, and the Audit tab shows which AI bots your robots.txt allows.