AI SEO: how to optimize your website for AI search engines

What does AI SEO mean?

AI SEO is optimizing your site for search products that answer with AI, so that they find, trust and cite your pages. The term is also used for writing SEO content with AI tools; that is a different topic, and not what this guide covers.

The engines involved are Google AI Overviews and AI Mode, ChatGPT search, Perplexity, Microsoft Copilot, Gemini and Claude. Each one runs searches, reads candidate pages and writes an answer with source links. AI SEO overlaps with what agencies call AEO and GEO; the label matters less than the checklist.

Google's own position is a useful anchor. Its AI optimization guide, updated 10 July 2026, says optimizing for generative AI search "is still SEO". So AI SEO is classic SEO done properly, plus a set of checks for crawlers and signals that classic SEO did not need to think about.

Which AI crawlers should you allow?

Allow the search and answer crawlers, and decide separately about the training crawlers. Blocking a search crawler removes you from that engine's answers; blocking a training crawler does not.

OpenAI documents that OAI-SearchBot surfaces sites in ChatGPT search, that sites opting out are not shown in search answers, and that robots.txt changes take about 24 hours. GPTBot only collects training data. Google-Extended is a control token, not a crawler: it governs Gemini training and grounding, and Google says it does not affect inclusion in Google Search. AI Overviews and AI Mode use Googlebot.

Perplexity and Anthropic document the same split. Perplexity says PerplexityBot surfaces sites in results and is not used for training, while Perplexity-User fetches pages for users and generally ignores robots.txt. Anthropic lists ClaudeBot for training, Claude-SearchBot for search quality and Claude-User for user-requested fetches, and says all three follow robots.txt. Apple's Applebot crawls for Siri and Spotlight, while Applebot-Extended only controls training use and does not crawl.

Is your CDN blocking AI search crawlers?

It may be, especially on Cloudflare. Check the dashboard and your server logs rather than trusting robots.txt alone, because a CDN rule can block a crawler that robots.txt allows.

Cloudflare changed its default on 1 July 2025 to block AI crawlers unless they pay, and in June 2026 said AI training crawlers would be blocked by default for all new domains; it reported that training makes up 52% of crawler requests. Its managed robots.txt can add a content signal such as search=yes, ai-train=no and disallow GPTBot, ClaudeBot, Google-Extended, Applebot-Extended and CCBot.

Cloudflare also published crawl-to-referral ratios of about 750 to 1 for OpenAI and about 30,000 to 1 for Anthropic, which is why many publishers block training. That is a reasonable choice, as long as the search crawlers stay open. Confirm in your logs that OAI-SearchBot, PerplexityBot and Claude-SearchBot receive HTTP 200 responses, not 403 or 402.

How should content be written for AI answers?

Write so a single passage can be lifted and still make sense: a question as the heading, a direct one or two sentence answer, then the detail. Engines quote passages, not whole pages.

Google says there is no requirement to break content into tiny pieces, so this is not about chopping pages up. It is about clarity. Name the subject in the sentence instead of writing "it", give numbers with their source, use tables for comparisons, and show a last-updated date. Cover the follow-up questions around your topic too, because Google's AI features may use query fan-out, running several related searches before choosing supporting links.

• Question H2s that match how buyers actually ask.
• A direct answer in the first one or two sentences of each section.
• Comparison tables, definitions and original data with sources named.
• Visible last-updated dates and current figures.
• Main content in server-rendered HTML, not only after JavaScript runs.

Does schema markup help with AI search?

Schema helps general SEO and keeps entity facts unambiguous, but it is not a special key to AI answers. Google says structured data isn't required for generative AI search and there's no special schema.org markup to add.

Use it where it already earns its place: Organization schema with sameAs links to your official profiles, Product or Service schema where accurate, and FAQ markup that matches visible text. Google requires structured data to match visible content. The same caution applies to llms.txt: no major answer engine has confirmed using it, and Google's John Mueller compared it to the meta keywords tag.

Why do off-site signals matter so much?

Because AI answers lean heavily on third-party sources, what other sites say about you can decide whether you are named. AI SEO therefore extends past your own domain.

Peec AI's analysis of 30 million sources puts Reddit, YouTube, LinkedIn, Wikipedia, Forbes, G2 and Yelp among the most-cited domains. Keep your brand facts consistent across those platforms and the directories in your industry, earn genuine reviews, and answer questions in communities under your real name. Google's guide advises against inauthentic mentions, so seeded posts are a risk, not a shortcut.

Start with the sources your own prompts already cite. If ChatGPT answers your category questions from two review sites and a Reddit thread, those are the places where accurate, current information about you will do the most good, and they make a far better brief than a generic link-building target list.

How do you measure AI SEO results?

Track three things: citation share on a fixed prompt set, first-party AI reports, and AI referral sessions. Rankings alone miss most of what happens in AI answers.

Clicks from classic results are under pressure, which is why rankings are no longer enough. Ahrefs found position-one click-through rate was 58% lower when an AI Overview was present, using December 2025 data. Search Console counts AI feature traffic in the Web search type and added a generative AI report in June 2026; Bing Webmaster Tools AI Performance shows Copilot citations and citation share; GA4 shows referrals from chatgpt.com, perplexity.ai, copilot.microsoft.com and gemini.google.com.

What does the checklist look like in practice?

Work in order: access, indexing, content, entities, mentions, measurement. Skipping to content before fixing access is a common and costly mistake.

Each step has a check you can verify, which keeps the work honest. Our AEO checklist post expands each line into the exact tests.

• Access: robots.txt and Cloudflare allow every search crawler; logs show 200 responses.
• Indexing: key pages indexed in Google and Bing, eligible for snippets, submitted via sitemaps and IndexNow.
• Content: question headings, direct answers, tables, dates, server-rendered HTML.
• Entities: Organization schema, sameAs, consistent facts on G2, LinkedIn, Google Business Profile.
• Mentions: reviews, Reddit, YouTube and editorial lists your prompts already cite.
• Measurement: fixed prompt set, Search Console, Bing AI Performance, GA4 referrals.

How does AEO Services Agency approach this work?

We fix SEO foundations first, then do the AI-specific work, and we measure everything against a baseline. AEO Services Agency is the AEO and GEO practice of Greenwolf Tech Labs.

Engagements start with a free AEO visibility check that includes a crawler and Cloudflare review, then run at a fixed price. We measure a fixed prompt set before and after changes and report citation share and AI referral traffic monthly. No one can promise a citation, so we promise the work and the measurement instead.