Shubi
ContactFree AI snapshot
Menu

Home / AI readiness check

Free AI readiness check

Can AI crawlers read your website?

Enter your website. We read up to 10 pages the way an AI crawler does, without JavaScript, and run 28 checks. The report appears on this page, with the evidence for every finding. Free, no sign-up.

What it checks

28 checks in 6 areas

Each check reports what was found, what was expected, and how it was tested. Checks that do not apply to your site are left out of the score.

AI crawler accessibility

  • Homepage can be fetched
  • robots.txt is served correctly
  • Search and AI answer crawlers are allowed by robots.txt
  • Model-training crawlers: stated policy
  • Site is served over HTTPS
  • Linked pages respond
  • Content is in the HTML without JavaScript

Technical discoverability

  • XML sitemap is available
  • Homepage is not marked noindex
  • Pages declare a canonical URL
  • Pages have unique titles
  • Pages have a meta description
  • Pages have one main heading
  • Page language is declared
  • Homepage has no redirect chain

Structured data

  • JSON-LD blocks are valid JSON
  • Organization structured data
  • WebSite structured data
  • Inner pages have breadcrumb data
  • Articles have Article data

Entity recognition

  • Official profiles are linked with sameAs
  • Brand name is consistent in page metadata
  • About and contact pages are linked from the homepage

Content extractability

  • Long pages are divided with subheadings
  • Articles show when they were published
  • llms.txt is present

Trust and reputation

  • Privacy policy and terms are linked
  • Articles name their author

What it does not tell you

  • It does not measure AI visibility. It asks no AI product about your brand, so it cannot say whether ChatGPT, Gemini or others mention, cite or recommend you.
  • It covers 55% of our scoring model by weight. Brand authority, topical authority, citations and competitors need other data and are shown as not tested.
  • The score weights are our own choices. They are not ranking factors of any search engine or AI product, and a high score does not guarantee visibility.
  • It reads robots.txt, not your firewall. A firewall can block a crawler that robots.txt allows.

To see how AI describes your brand, request the free AI visibility snapshot. How we measure that is on How we measure.

About ShubiBot

The check fetches pages with the user agent ShubiBot. It runs only when someone enters a website on this page.

  • It obeys robots.txt. To keep it out, add User-agent: ShubiBot and Disallow: /.
  • It reads at most 10 pages per check, two at a time, and does not run JavaScript, submit forms or log in.
  • It fetches public pages only, and we do not store the pages or the report.

See how AI describes your brand today.

Get a free AI visibility snapshot: 10 real buyer questions, 5 AI engines, scored against your competitors.

Get your free snapshotSee pricing