68 Million AI Crawler Visits Reveal What Drives AI Search Visibility — Plus the Ghost Citation Problem

A study published by Search Engine Journal on April 20, 2026, analyzed 68.9 million AI crawler visits across 858,457 websites during February 2026 — the most granular public look at AI crawler behavior yet. Separately, Kevin Indig's research across 3,981 domains reveals that 62% of all AI citations are ghost citations where the brand gets a link but zero name recognition in the answer text.

Together, these studies reshape what AI SEO actually means in practice: it's no longer about whether AI crawlers find you, but whether they credit you when they use your content.

1. The AI Crawler Scene: 68.9 Million Visits in

One Month The most significant finding is the shift in *why* AI bots crawl. The majority of AI crawler traffic is no longer about building training datasets. Instead,

56.9% of all AI crawler activity (39.8 million visits) is classified as User Fetch — real-time content retrieval triggered by a live user query in ChatGPT, Perplexity, or similar AI search interfaces. This aligns with trends covered in our analysis of AI crawler visit patterns and Stanford's adoption research , the shift from training crawlers to real-time retrieval bots is accelerating. ## 2. Who Is Crawling: OpenAI Owns 81% of AI Bot Traffic The concentration of AI crawler traffic is extreme. OpenAI accounts for 81.0% of all AI crawler visits (55.8 million out of 68.9 million), making it the dominant force in AI web crawling by an enormous margin. Google's low crawler volume is notable. At just 380,000 visits (0.6%), Gemini's crawling footprint is 147 times smaller than OpenAI's. This likely reflects Google's ability to use its existing Googlebot index rather than deploying separate AI-specific crawlers.

Year-Over-Year LLM Referral Traffic Growth

Referral traffic from LLM-powered search is growing rapidly, with some platforms showing explosive growth:

3. What Drives AI Crawl Rates: Integrations, Schema, and

Content Depth The study isolates three categories of signals that predict higher AI crawl rates. Each contributes independently, and the effect compounds when combined.

Third-Party Integrations

Sites with Yext integration achieved a 97.1% crawl rate, meaning nearly every site was visited by at least one AI crawler. The likely mechanism: Yext syndication distributes business data across the web, creating more reference points for AI systems to discover and validate.

Structured Data and Business Profile Signals

The granularity of structured data matters. Sites with no schema fields completed had a 55.2% crawl rate. Sites with 10–11 fields completed reached 82% , a 26.8 percentage point improvement. Each additional completed schema field adds roughly 2.7 percentage points of crawl probability. This reinforces findings from our Cloudflare Agent Readiness Score analysis on structured data's role in AI visibility.

Content Depth:

The 33x Multiplier Content volume is the single strongest predictor of AI crawler visit frequency:

This 33x difference is the largest effect size in the entire study, reinforcing that AI systems disproportionately target content-rich sites for real-time retrieval. ## 4. Business Impact: Crawled Sites Get 3.2x More Traffic The study goes beyond crawl rates to measure business outcomes. Sites that received AI crawler visits consistently outperformed uncrawled sites:

---

5. The Ghost Citation Problem: 62% of AI Citations Never Name You

Even if you win AI crawler attention and earn a citation in AI-generated answers, a separate problem looms: the AI probably won't mention your brand by name. Research from Kevin Indig, published in Growth Memo on April 21, 2026, quantifies what he calls the ghost citation problem.

The study tested four AI search engines , ChatGPT, Google AI Overviews, Gemini, and Google AI Mode , and found that 62% of all citations are ghost citations. A ghost citation occurs when the AI includes a source link but never mentions the brand name in the answer text. The mechanism is structural, not random. Informational content (articles, guides, how-to pages) is the most vulnerable to ghost citation because the AI extracts facts without needing to endorse the source. Comparative and evaluative content ("best X for Y", product reviews, tool comparisons) generates brand mentions because the AI must name the entities being compared. This connects directly to the ChatGPT citation mechanics study showing only 1.93% of Reddit pages get cited despite heavy retrieval.

6. Platform Comparison: How Each AI Engine Handles Citations

Each AI search engine has a distinct citation personality, and understanding these differences is critical for prioritizing your GEO strategy.

ChatGPT and Gemini are near-opposites. ChatGPT cites sources 87% of the time but only names brands 20.7% of the time , it gives you the link but rarely the brand visibility. Gemini does the reverse: it mentions brand names 83.7% of the time but only provides a clickable citation link 21.4% of the time.

Geographic Variation in Brand Mentions Brand mention rates vary significantly by country, which matters for

international SEO strategy: The cross-engine disagreement rate is also notable: 22% of 454 prompt-domain combinations produced different mention outcomes across engines, meaning the same brand is named by one AI and ghosted by another for the same query.

7. Action Plan: Optimizing for Both AI Crawling and AI Citations

Combining findings from both studies, here is a concrete plan for improving both AI crawler visibility and brand citation quality.

For AI Crawl Visibility

For Brand Citation Quality

Frequently Asked Questions