For Google AI Overviews and AI Mode, ChatGPT search, Perplexity, Gemini or Copilot to find and cite your website, it first has to be technically accessible: AI search crawlers are not blocked, your key pages are in the Google and Bing indexes, and the main text is visible in the HTML without JavaScript. The second step is a clear page structure, visible dates, an author and consistent business details everywhere. Below is a 15-point checklist with explanations and links to each search engine’s official documentation.
This checklist covers technical readiness. How to write content and why mentions on other websites matter is explained in our article “How to get found in ChatGPT and Google AI answers”.
The checklist: what should you check?
Start with points 1–8: if a crawler cannot reach a page or the page is not indexed, structure and markup will not help.
| No. | What to check | How to check | Where it matters |
|---|---|---|---|
| 1 | OAI-SearchBot is not blocked | open yoursite.lt/robots.txt | ChatGPT search |
| 2 | PerplexityBot is not blocked | robots.txt | Perplexity |
| 3 | Claude-SearchBot and Claude-User are not blocked | robots.txt | Claude |
| 4 | Googlebot and Bingbot can reach key sections, CSS and JS | robots.txt, URL Inspection in Search Console | Google, Bing, Copilot |
| 5 | Key pages are in Google’s index | Search Console > URL Inspection | AI Overviews, AI Mode, Gemini |
| 6 | Site verified in Bing, sitemap.xml submitted | Bing Webmaster Tools | Bing, Copilot |
| 7 | No unnecessary noindex or nosnippet tags | page source, URL Inspection | Google AI features |
| 8 | Main text, prices and contact details are in the HTML | “View page source” in your browser | all AI crawlers |
| 9 | A short answer at the top of the page | read the first paragraph | all engines |
| 10 | Headings phrased as questions, comparisons in tables | review your 5 most important pages | all engines |
| 11 | Organization and Article markup without errors | Rich Results Test | |
| 12 | Visible publication and update dates | the page and datePublished / dateModified | |
| 13 | An author or responsible team is named | article pages | all engines |
| 14 | Consistent business name, address and phone number | website, profiles, directories | all engines |
| 15 | LCP, INP and CLS in the good range | PageSpeed Insights | Google, visitors |
Can AI search crawlers reach your website?
Open yoursite.lt/robots.txt and check that there are no “Disallow” rules for AI search crawlers. Most companies separate their search crawlers from their model-training crawlers, so you can opt out of training without losing visibility in search.
| Crawler | Company | Used for | What blocking means |
|---|---|---|---|
| OAI-SearchBot | OpenAI | surfacing websites in ChatGPT search | pages are not shown in ChatGPT search answers (navigational links may remain) |
| GPTBot | OpenAI | content for model training | no effect on ChatGPT search |
| ChatGPT-User | OpenAI | opening a page at a user’s request | robots.txt rules may not apply |
| PerplexityBot | Perplexity | surfacing and linking websites in Perplexity search; not used for training | your site drops out of Perplexity search results |
| Claude-SearchBot | Anthropic | improving search result quality | visibility in Claude search may fall |
| ClaudeBot | Anthropic | content for model training | signals that content should be excluded from training |
| Googlebot | Google Search, including AI Overviews and AI Mode | pages disappear from Search and its AI features | |
| Google-Extended | not a separate crawler but a token: whether content may be used for Gemini models and grounding their answers | no effect on inclusion or ranking in Google Search | |
| Bingbot | Microsoft | Bing search | pages are not added to Bing’s index |
Sources: OpenAI, Perplexity, Anthropic, Google, Bing.
An example that blocks training but allows search:
User-agent: GPTBot
Disallow: /
User-agent: ClaudeBot
Disallow: /
User-agent: *
Allow: /
Sitemap: https://yoursite.lt/sitemap.xml
- Only block Google-Extended deliberately. According to Google, the token also covers grounding in the Gemini apps, where content from the Search index is supplied to the model.
- Firewall. If your server or CDN has bot protection switched on, check that it does not block the search crawlers you allow in robots.txt.
- User-request agents (ChatGPT-User, Perplexity-User, Claude-User) open a page when someone asks for it in a chat, so they treat robots.txt differently. See each company’s documentation.
Are your pages in the Google and Bing indexes?
Google AI Overviews and AI Mode have no extra requirements: Google Search Central says a page must be indexed and eligible to appear in Search with a snippet. So the first check is ordinary indexing.
- Google. Check your home page and 3–5 key pages with the URL Inspection tool in Search Console. The nosnippet, data-nosnippet, max-snippet and noindex controls limit what can be shown from a page, including in AI features.
- Gemini. Because content from Google’s Search index can be supplied to Gemini for its answers, the same indexing matters here too.
- Bing and Copilot. Verify your site in Bing Webmaster Tools and submit your sitemap.xml. Since February 2026 it has included the AI Performance report, which shows which of your pages are cited in Microsoft Copilot, Bing AI summaries and selected partner integrations.
- IndexNow. Bing offers this protocol so that participating search engines learn about new, updated or removed pages straight away.
Is your text visible without JavaScript?
Your main text, prices and contact details should be in the HTML the server returns. Google renders JavaScript in a separate step after the page waits in a queue, and it notes that not all bots can run JavaScript, and that server-side rendering or pre-rendering is still a good idea. Other AI crawlers’ documentation does not say whether they execute JavaScript, so it is safest not to depend on it.
How to check: open the page, press Ctrl+U (“View page source”) and search for a sentence from the main text. If it is missing and you only see an empty container and script links, an AI crawler probably sees the same.
How should you structure a page so it is easy to cite?
Google says AI features need no special optimisation: the same principles of helpful, people-first content apply. In practice that means a clear structure that both people and search engines can read easily.
- A short answer first. The first paragraph should answer the main question in 2–4 sentences, with the detail below.
- Headings as questions. “How much does … cost?”, “How long does … take?”, “What is the difference between …?”, the way your customers ask.
- Tables. Prices, timeframes, comparisons and specifications belong in a table, not scattered through the text.
- One page, one topic. If the same service is described on three pages, it is unclear which one to cite.
- Specific facts. A “from” price, a timeframe, an address and the steps of the work instead of generic phrases.
Do dates and authorship matter?
Google recommends showing a visible date labelled “Published” or “Updated” and using the same date in structured data (datePublished, dateModified). At the same time, Google lists changing a page’s date without substantial content changes as a sign of search-engine-first content. In the same guidance, Google encourages accurate authorship information, such as a byline linking to details about the author or team.
Are your business details consistent everywhere?
Your business name, address, phone number (NAP) and company code should match on your website, business profiles, company directories and social media. If different places show different addresses or an old phone number, outdated details may end up in an AI answer. Check your contact page, footer, structured data and at least 3–5 external profiles.
Which structured data do you need?
No special schema.org markup is needed for AI features; Google says so directly. Standard markup still helps a search engine understand your organisation and your articles, and the key rule is that markup must only describe what is visible on the page.
| Type | Where to use it | What to include |
|---|---|---|
| Organization (or LocalBusiness) | home page | name, logo, address, contact details, sameAs links to your profiles |
| Article | articles | headline, author, datePublished, dateModified, image |
| BreadcrumbList | all inner pages | breadcrumb trail |
| FAQPage | pages with an FAQ block | only the questions and answers visible on the page |
According to Google, Organization markup helps distinguish your business from other organisations with similar names. An important caveat about FAQPage: in September 2023 Google limited FAQ rich results to well-known government and health websites, and from 7 May 2026 stopped showing them in Search altogether (documentation updates). An FAQ block on the page is still useful for visitors, but expandable questions will not appear in Google results.
Do you need an llms.txt file?
llms.txt is a file proposed by Jeremy Howard in September 2024. It sits at the site root (/llms.txt) and uses Markdown to briefly describe the website and its key pages so that AI agents can use the site more easily. It is a proposal, not an adopted standard.
- None of the search engines mentioned here has confirmed that it uses llms.txt for answers or citations.
- Google states plainly that you do not need to create new machine-readable files or AI text files to appear in AI Overviews and AI Mode.
- The file does no harm and is quick to create for a small website, so you can have one, but do not expect a noticeable effect.
Does speed matter?
Speed matters first of all to your visitors and to Google Search. Assess the Core Web Vitals measured on real visitors’ devices: a page is in good shape when, for 75% of visits, LCP is within 2.5 s, INP within 200 ms and CLS within 0.1. Check the mobile version, as Google indexes the mobile version of pages.
How can we help?
Saitai.lt checks your website against this list and fixes the obstacles it finds: robots.txt, indexing in Google and Bing, structured data, page structure and consistency of your business details.
- AI search optimisation (GEO): ongoing work so that AI search finds and mentions your business, from €349/month.
- On-page optimisation: a one-off clean-up of page structure, headings and markup, from €119.
- If your text only appears with JavaScript or your system is outdated, website modernisation can help. All related services are under “Search visibility”.
If you want AI to answer visitors on your own website, see AI search for websites; prices for all AI solutions are on the AI solutions cost page.
Written by Saitai.lt team. Published , updated .
Sources
- Google Search Central: AI features and your website (external website)
- Google Search Central: Google common crawlers (Google-Extended) (external website)
- OpenAI: Overview of OpenAI Crawlers (external website)
- Perplexity: Perplexity Crawlers (external website)
- Anthropic: Does Anthropic crawl data from the web, and how can site owners block the crawler? (external website)
- Bing Webmaster Tools: Which crawlers does Bing use? (external website)
- Bing Webmaster Blog: Introducing AI Performance in Bing Webmaster Tools (February 2026) (external website)
- Google Search Central: Understand JavaScript SEO basics (external website)
- Google Search Central: Creating helpful, reliable, people-first content (external website)
- Google Search Central: Add a byline date to Google Search results (external website)
- Google Search Central: Organization structured data (external website)
- Google Search Central: Article structured data (external website)
- Google Search Central: documentation updates (FAQ rich results) (external website)
- The llms.txt proposal (Jeremy Howard, 2024) (external website)
- web.dev: Web Vitals (external website)
- Google Search Central: Mobile-first indexing best practices (external website)