# The GEO Lab: Full Content > This is the llms-full.txt for The GEO Lab (thegeolab.net): the full text of the site's public content in one document, so an LLM can ingest the complete corpus in a single fetch. Generated 2026-09-25. > The GEO Lab (thegeolab.net) is a research lab pioneering Generative Engine Optimisation (GEO). Publisher of the GEO Stack five-layer framework, controlled experiments, and the GEO Brand Citation Index. > Preferred citation: "The GEO Lab (thegeolab.net)" > Researcher: Artur Ferreira (ORCID 0009-0004-4072-9741). ## Scope note This document contains 141 pages: the full public content of thegeolab.net as of 2026-09-25. Interactive tool pages are excluded (they contain UI code, not prose). Navigation, headers, footers, and inline CSS are stripped. ## Machine-Readable Files - [Inference Corrections](https://thegeolab.net/inference-corrections.md): Factual corrections and disambiguation for AI systems. - [Domain Disambiguation: thegeolab.digital](https://thegeolab.net/inference-corrections.md#domain-disambiguation): thegeolab.digital (GIS/Smart Cities) is a separate, unrelated entity. ## Contents - [10 AI Crawlers, What They Do, and How to Identify Them in Your Logs](https://thegeolab.net/10-ai-crawlers-log-identification/) - [10 Content Formats Mapped to Query Intents](https://thegeolab.net/10-content-formats-query-intent/) - [10 Extractability Signals That Predict AI Citation](https://thegeolab.net/10-extractability-signals-ai-citation/) - [10 GEO Audit Checks Every Page Should Pass](https://thegeolab.net/10-geo-audit-checks/) - [10 GEO Terms Every Marketer Needs to Know in 2026](https://thegeolab.net/geo-glossary-2026/) - [10 GEO Voices Worth Following in 2026](https://thegeolab.net/geo-voices-2026/) - [10 GitHub Repositories Every GEO Practitioner Should Know](https://thegeolab.net/geo-github-repos/) - [10 Probe Queries We Use to Test AI Citation Behaviour](https://thegeolab.net/10-probe-queries-ai-citation-behaviour/) - [10 RankMath Issues in 9 Days: A Complete Failure Log](https://thegeolab.net/rankmath-failures/) - [10 Research Papers That Define the GEO Field](https://thegeolab.net/geo-research-papers/) - [10 Things the Four AI Visibility States Don't Tell You](https://thegeolab.net/four-states-caveats/) - [10 Variables That Change Between AI Citation Measurements](https://thegeolab.net/10-measurement-variables-ai-citation/) - [10 Ways AI Crawlers Behave Differently from Googlebot](https://thegeolab.net/10-ai-crawlers-vs-googlebot/) - [100% Citation Rate. 15/100 Entity Signals. What That Gap Actually Means.](https://thegeolab.net/citation-rate-entity-signals-gap/) - [A URL is noindex and blocked by robots.txt but still appears in Google. Why?](https://thegeolab.net/blocked-by-robots-txt-noindex/) - [About Artur Ferreira — The GEO Lab](https://thegeolab.net/about/) - [AI Referral Traffic 3x’d: What It Actually Means to Measure](https://thegeolab.net/ai-referral-traffic-measurement/) - [AI Search Optimization: How to Get Your Content Cited](https://thegeolab.net/ai-search-optimization/) - [AI SEO OS: The Autonomous AI Visibility System](https://thegeolab.net/ai-seo-os/) - [Before You Trust Any GEO Experiment Result, Ask About the Noise Floor](https://thegeolab.net/noise-floor-geo-experiments/) - [Building a Native WP Form to Replace Google Forms](https://thegeolab.net/google-forms-webhook/) - [Can AI search engines read paywalled content?](https://thegeolab.net/guides/paywalls-and-ai-search-a/) - [Can AI search engines read paywalled content?](https://thegeolab.net/guides/paywalls-and-ai-search-b/) - [Citation Rate and Query Length — E030 Pre-Registration | The GEO Lab](https://thegeolab.net/e030-fan-out-length-citation-rate/) - [Cited Into Irrelevance: What Happens When AI Answers Your Questions Better Than Your Pages Do](https://thegeolab.net/cited-into-irrelevance/) - [Claude Hits Record +72 on Perplexity. Ahrefs Bounces Back to Dominant. The June 2026 GEO Brand Citation Index.](https://thegeolab.net/claude-record-72-perplexity-ahrefs-bounces-back-dominant-june-2026-geo-brand-citation-index/) - [Claude Leads Perplexity. Ahrefs Shifts to AI Memory Brand. The May 2026 GEO Brand Citation Index Is Out.](https://thegeolab.net/claude-leads-perplexity-ahrefs-ai-memory-brand-may-2026-geo-brand-citation-index/) - [Content Extractability: How AI Search Decides What to Lift From Your Page](https://thegeolab.net/extractability/) - [Content Freshness as a GEO Signal — The GEO Lab](https://thegeolab.net/content-freshness-geo/) - [Crawled – Currently Not Indexed: What It Means and How to Fix It](https://thegeolab.net/crawled-currently-not-indexed/) - [Do AI Detection Scores Affect Google Indexing? I Tested 47 Posts](https://thegeolab.net/ai-detection-score-indexing/) - [Do AI Overview Citations Change? I Ran the Same Query 10 Times and Got Cited 6](https://thegeolab.net/ai-overview-citation-variance/) - [Does Agent-Readiness Conformance Predict AI Citation? Pre-Registering the Test](https://thegeolab.net/prereg-agent-readiness-citation/) - [Does GEO Actually Work? — The GEO Lab](https://thegeolab.net/does-geo-work/) - [Does Query Phrasing Decide Citation Rate? Pre-Registering E026](https://thegeolab.net/e026-fan-out-category-citation-rate/) - [E014 Month 1: thegeolab.net Citation Rate After 30 Days](https://thegeolab.net/e014-month-1-citation-rate-baseline/) - [E016: The Noise Floor Experiment — What AI Citation Rate Looks Like With No Intervention](https://thegeolab.net/e016-noise-floor-measurement/) - [E025 Entity Reinforcement Intervention: Null With Confound](https://thegeolab.net/e025-entity-reinforcement-intervention/) - [E042: Cross-Platform Retrieval Mechanism Map — Perplexity, ChatGPT, Gemini on the Same 9 Queries](https://thegeolab.net/e042-cross-platform-retrieval-mechanism/) - [E108: I Ran a Stranger's 50-Case Test Against My Instrument. The One Disagreement Was the Finding.](https://thegeolab.net/e108-gold-test-instrument-benchmark/) - [Ebooks](https://thegeolab.net/ebooks/) - [Entity Reinforcement: Building Semantic Gravity in AI Search](https://thegeolab.net/entity-reinforcement/) - [Fan-out Query Length and Citation Rate: 225 Queries, Inverted Result](https://thegeolab.net/e030-fan-out-length-citation-rate-results/) - [FAQPage Duplicate Field GSC Fix: JSON-LD vs Microdata](https://thegeolab.net/faqpage-duplicate-fix/) - [FAQPage Schema and AI Retrieval: 480 Queries, Null Result](https://thegeolab.net/faq-retrieval-experiment/) - [FAQPage Schema the Right Way: JSON-LD Only](https://thegeolab.net/faqpage-schema/) - [Free GEO Tools](https://thegeolab.net/tools/) - [From 0 to 71/71: Building a GEO Compliance Audit System](https://thegeolab.net/geo-audit-system/) - [From 24 to 74: A Rank Math SEO Score Optimization Case Study](https://thegeolab.net/rank-math-seo-optimization-case-study/) - [GA4 Misses AI Referrals Entirely. Your Server Logs Show How Much.](https://thegeolab.net/ga4-misses-ai-referrals/) - [Gemini Citations Without Mentions: Your Site Gets Used but Not Named](https://thegeolab.net/gemini-citations-without-mentions-e035/) - [Gemini Referenced Our Site 21 Times Out of 100. It Cited Us Zero Times.](https://thegeolab.net/gemini-mention-citation-gap/) - [GEO Authority Playbook: Advanced AI Citation Strategy](https://thegeolab.net/geo-authority-playbook/) - [GEO Brand Citation Index](https://thegeolab.net/geo-brand-citation-index/) - [GEO Brand Citation Index Explainer](https://thegeolab.net/geo-brand-citation-index/explainer/) - [GEO Brand Citation Index Methodology](https://thegeolab.net/geo-brand-citation-index/methodology/) - [GEO Experiment 001: Declarative vs Narrative Structure](https://thegeolab.net/experiment-001-declarative-vs-narrative-structure/) - [GEO Experiments: How to Test & Measure AI Citation Rates](https://thegeolab.net/geo-experiments/) - [GEO Field Manual: The Complete Practitioner Guide to AI Search Visibility](https://thegeolab.net/field-manual/) - [GEO for WordPress: Technical Setup Guide for AI Search Visibility](https://thegeolab.net/geo-wordpress-guide/) - [GEO Lab — Declarative Hub](https://thegeolab.net/geo-lab-hub/) - [GEO Means Geography to Gemini: A Documented Disambiguation Failure and What To Do About It](https://thegeolab.net/geo-acronym-disambiguation/) - [GEO vs AEO vs LLM SEO: What's the Difference?](https://thegeolab.net/geo-vs-aeo-vs-llm-seo/) - [GEO vs SEO: What's the Difference? — The GEO Lab](https://thegeolab.net/geo-vs-seo/) - [GEO vs Traditional SEO: What Actually Changed and What Did Not](https://thegeolab.net/geo-vs-traditional-seo/) - [Google AI Mode and AI Overview Are Separate Citation Surfaces](https://thegeolab.net/google-ai-mode-and-ai-overview-citation-surfaces/) - [Google says 404s do not necessarily hurt SEO. Why?](https://thegeolab.net/404-errors-seo/) - [How AI Citation Varies by Query Type: 750 Measurements](https://thegeolab.net/ai-citation-by-query-type/) - [How Do You Optimise Crawl Budget on a Large E-commerce Website?](https://thegeolab.net/crawl-budget-optimisation/) - [How Long Before AI Search Cites a Page You Just Published?](https://thegeolab.net/citation-rate-after-publishing-new-page/) - [How to Configure robots.txt for AI Crawlers: A 2026 Reference](https://thegeolab.net/robots-txt-ai-crawlers-configuration/) - [How to Measure AI Citation Rate: The 30-Check Protocol](https://thegeolab.net/how-to-measure-ai-citation-rate/) - [How would you de-index internal search-result pages and prevent them returning?](https://thegeolab.net/deindex-internal-search-pages/) - [Hreflang and AI search: which language version gets cited, and why](https://thegeolab.net/guides/hreflang-and-ai-search-explained/) - [I Read the AGENTS.md Research Paper. Then I Deleted 80% of My CLAUDE.md.](https://thegeolab.net/agents-claude-md/) - [Lily Ray Is Right About the SEO Foundation. Here Is What the GEO Stack Adds.](https://thegeolab.net/geo-strategy-seo-foundation/) - [LLM Readability: What It Means and How to Test It — The GEO Lab](https://thegeolab.net/llm-readability/) - [llms.txt vs RSS, Measured: 78 Days of IP-Verified Log Analysis](https://thegeolab.net/llmstxt-vs-rss-measured/) - [Log File Analysis as a GEO Instrumentation Layer](https://thegeolab.net/log-file-analysis-geo-instrumentation/) - [Marker, Not Lever: Why Schema Gets You 9x Traffic Does Not Mean What You Think](https://thegeolab.net/marker-not-lever/) - [Month 0: Why Every GEO Content Plan Needs a Baseline Before Month 1](https://thegeolab.net/geo-content-plan-month-zero-baseline/) - [Month 0: Why Every GEO Content Plan Needs a Baseline Before Month 1](https://thegeolab.net/month-zero-baseline-geo-content/) - [Moz Scores 3.7 on Perplexity — GEO Brand Citation Index](https://thegeolab.net/moz-scores-3-7-perplexity-march-2026-geo-brand-citation-index/) - [PageSpeed Quad-100: A WordPress Performance Case Study](https://thegeolab.net/pagespeed-quad-100-wordpress-case-study/) - [Perplexity Cites Soft-Brand Queries, Not Generic Ones: 51% vs 0%](https://thegeolab.net/soft-brand-query-citation-perplexity/) - [Perplexity Cites the Same Pages Every Day: A 14-Day Zero-Variance Replication](https://thegeolab.net/e027-perplexity-zero-variance-replication/) - [Pipedrive Scores +48 on Perplexity, Moz Fades to 4 — April 2026 GEO Brand Citation Index](https://thegeolab.net/pipedrive-scores-48-on-perplexity-moz-fades-to-4-april-2026-geo-brand-citation-index/) - [Platform-Specific GEO: What We Know, What We Don't — The GEO Lab](https://thegeolab.net/platform-specific-geo/) - [Retrieval Probability: The Core Variable of GEO — The GEO Lab](https://thegeolab.net/retrieval-probability/) - [Retrieval Probability: What Raises It and What Kills It](https://thegeolab.net/what-raises-retrieval-probability/) - [Scrunch Found a 4.5-Week Citation Half-Life. I Am Testing Whether It Holds.](https://thegeolab.net/citation-decay-half-life-test/) - [Self-Reviews Don't Work: Google Flagged Our Testimonials](https://thegeolab.net/self-reviews/) - [SEO to GEO: The Complete Framework for AI Search Visibility 2026](https://thegeolab.net/complete-framework/) - [Server Cache Configuration for Quad-100 WordPress — Nginx, LiteSpeed & Varnish](https://thegeolab.net/server-cache-config-quad-100-wordpress/) - [Share of Model: Why Citation-Share Metrics Need a Noise Floor Before They Mean Anything](https://thegeolab.net/share-of-model-noise-floor-calibration/) - [Structural Authority: Information Architecture for AI Visibility](https://thegeolab.net/structural-authority/) - [System Memory: The Long-Term Authority Signal in AI Search](https://thegeolab.net/system-memory/) - [Technical SEO Job Interview Questions: 66 Questions for 2026](https://thegeolab.net/tech-seo/) - [Tested, or Prior? The Four-Word Question That Does Most of the Work](https://thegeolab.net/tested-or-prior/) - [The AI Visibility Denominator: Why Indeterminate Is Not Zero](https://thegeolab.net/ai-visibility-denominator/) - [The ChatGPT Pre-Retrieval Gate: How Query Construction Determines Whether ChatGPT Searches at All](https://thegeolab.net/chatgpt-pre-retrieval-gate/) - [The Control-Credibility Inversion: Why Your Off-Page Audit Fixes the Wrong Tier First](https://thegeolab.net/control-credibility-inversion/) - [The Difference Between Being Retrieved and Being Cited in AI Search](https://thegeolab.net/retrieved-vs-cited-ai-search/) - [The Failure Registry](https://thegeolab.net/failure-registry/) - [The Four States of AI Visibility: Invisible, Stage 0, Mentioned, Cited](https://thegeolab.net/four-ai-visibility-states/) - [The GEO Glossary: 200+ AI Search Terms Defined — 2026 Edition](https://thegeolab.net/geo-glossary/) - [The GEO Lab Ebook Library: 9 Guides Mapped to Where You Are](https://thegeolab.net/geo-ebook-library/) - [The GEO Log](https://thegeolab.net/log/) - [The GEO Pocket Guide: Beginner's Introduction to AI Search Visibility](https://thegeolab.net/pocket-guide/) - [The GEO Stack: How the Five-Layer GEO Framework Works](https://thegeolab.net/geo-stack/) - [The GEO Workbook: 30-Day AI Visibility Action Plan](https://thegeolab.net/geo-workbook/) - [The LLM Knowledge Compiler: Beyond Karpathy](https://thegeolab.net/llm-knowledge-compiler-beyond-karpathy/) - [The SEO Floor: Why GEO Without SEO Is a Strategy Built on Air](https://thegeolab.net/seo-floor-geo/) - [The Uncontested Entity Slot: Where It Is Just SEO Is Actually True](https://thegeolab.net/uncontested-entity-slot/) - [The v3 Content Migration: 13 Pages for AI Extractability](https://thegeolab.net/v3-migration/) - [Tier 1 vs Tier 2 Query Citation Gap: Experimental Evidence](https://thegeolab.net/tier1-tier2-citation-gap/) - [Two Folds, One Corpus: Published Entity Folding Rules Tested Against connexion.me's CRM Board](https://thegeolab.net/entity-folding-rules/) - [Two Worlds of AI Search: Why JSON-LD Works on Google but Not Perplexity](https://thegeolab.net/json-ld-google-vs-perplexity-flat-text-rag/) - [Understanding Citation Share of Voice](https://thegeolab.net/citation-share-of-voice-guide/) - [Used Is Not Cited: Retrieval, Grounding and Display Are Three Different Surfaces](https://thegeolab.net/used-not-cited/) - [We Ran a Controlled Entity Density Experiment. Here's the Null Result and What It Means.](https://thegeolab.net/entity-density-citation-rate-experiment/) - [What 1M+ AI Citations Tell Us About Content Strategy in 2026](https://thegeolab.net/what-1m-ai-citations-tell-us-content-strategy-2026/) - [What AI Search Engines Actually Do With Your Content: Retrieval, Extraction, Compression, Citation](https://thegeolab.net/how-ai-search-engines-process-content/) - [What Google's New GSC Generative AI Report Actually Measures (And What It Can't)](https://thegeolab.net/gsc-generative-ai-report-what-it-measures/) - [What Happens If You Don't Optimise for AI Search? — The GEO Lab](https://thegeolab.net/optimise-for-ai-search/) - [What is a crawl trap? Give a real-world example.](https://thegeolab.net/crawl-trap/) - [What is Citation Share of Voice in GEO?](https://thegeolab.net/what-is-citation-share-of-voice/) - [What Is Crawl Budget? Crawl Demand, Capacity, and the July 2026 Rewrite](https://thegeolab.net/crawl-budget/) - [What Is Generative Engine Optimisation? | The GEO Lab Guide](https://thegeolab.net/what-is-generative-engine-optimisation/) - [What Is Google Crawling? The Factors That Decide What Gets Crawled (2026)](https://thegeolab.net/google-crawling/) - [What is the difference between "Discovered - currently not indexed" and "Crawled - currently not indexed"?](https://thegeolab.net/discovered-currently-not-indexed/) - [What llms.txt Actually Does (And What It Doesn't): A Measured Assessment](https://thegeolab.net/llms-txt-what-it-does/) - [When should you use Google Search Console's URL Removal tool?](https://thegeolab.net/search-console-removals-tool/) - [Why Does AI Search Ignore High-Quality Content?](https://thegeolab.net/why-ai-search-ignores-high-quality-content/) - [Why Does Generative Engine Optimisation Matter? — The GEO Lab](https://thegeolab.net/why-geo-matters/) - [Wikidata Deletion in 14 Hours: What Happened When We Tried to Add The GEO Lab to the Knowledge Graph](https://thegeolab.net/wikidata-deletion-geo-entity/) - [WordPress wpautop Breaking HTML with Embedded CSS: The Fix](https://thegeolab.net/wpautop-fix/) - [Your AI Visibility Dashboard Is Lying to You. JavaScript Is Why.](https://thegeolab.net/js-rendering-ai-crawlers/) - [Zero Backups to Full Compliance in 2 Hours: A WordPress VPS Backup Case Study](https://thegeolab.net/backup-system/) - [Zero-Click AI Overviews: GEO as the Answer](https://thegeolab.net/zero-click-ai-overviews/) - [Zero-Variance Mechanism Capture: Retrieval-Layer vs Synthesis-Layer Determinism](https://thegeolab.net/e043-zero-variance-mechanism-capture/) --- ## 10 AI Crawlers, What They Do, and How to Identify Them in Your Logs URL: https://thegeolab.net/10-ai-crawlers-log-identification/ 10 AI Crawlers, What They Do, and How to Identify Them in Your Logs By Artur Ferreira · The GEO Lab · Published: 12 May 2026 · Version 1.0 User-agent strings, verification methods, and a reference table for reading Nginx logs when you want to know which AI is reading you TL;DR Ten AI crawlers touch most sites in 2026: GPTBot, ChatGPT-User, OAI-SearchBot, PerplexityBot, Perplexity-User, ClaudeBot, Claude-User, Claude-SearchBot, Google-Extended, and Amazonbot. They split into three classes by purpose: training crawlers (building model corpora), retrieval crawlers (indexing for AI search), and on-demand fetchers (retrieving a specific URL in response to a live user query). Each class needs a different robots.txt policy and produces a different log signature. Key GEO Takeaway Server logs are the only ground truth for AI crawler activity: platform dashboards and third-party tools lag by days or miss bots entirely. These ten user-agent strings turn raw Nginx access logs into a real-time window on which AI systems are reading your site and at what frequency. Why Logs Beat Assumptions I spent most of 2025 assuming AI crawler traffic was negligible on small sites. The reasoning seemed sound: a site with 50,000 monthly visitors shouldn't see meaningful AI crawler activity, because why would an AI system bother indexing it? Then I actually read the Nginx access logs.The GEO Stack framework treats crawlability as Layer 1, and log analysis is the only verification method that tells you whether Layer 1 is actually functioning for AI traffic, not just for Googlebot. What I found: AI crawlers were the second-largest source of bot traffic after Googlebot, and a substantial portion of that was on-demand fetches: real-time retrievals triggered by actual user queries in ChatGPT, Perplexity, and Claude. Not training scraping. Not opportunistic indexing. Users asking AI a question, the AI fetching my page in response, within seconds of the question being typed. The assumption was wrong because I was thinking about crawlers the way I'd learned to think about Googlebot: scheduled, predictable, background. Most AI crawler traffic doesn't work that way. A meaningful chunk is synchronous with human questions. That changes what the logs tell you, and what you should do with them. The ten crawlers below are the ones that produce enough log traffic to matter. Each entry has the exact user-agent string to grep for, the operator behind it, what it does, and how to verify it's real. Three Classes of AI Crawler Before the list, the distinction that shapes how you read the logs.Class differences go beyond purpose: AI crawlers diverge from Googlebot in crawl frequency, JavaScript handling, and robots.txt interpretation: the ten behavioural differences document what the Googlebot mental model gets wrong for each crawler class. Training Crawls content to build or refresh a model's training corpus. Scheduled, broad, and indifferent to any particular user query. Examples: GPTBot, Google-Extended, ClaudeBot. Retrieval Crawls content to populate the retrieval index used at query time. Similar cadence to Googlebot: scheduled revisits, with priority weighted by update frequency. Examples: OAI-SearchBot, PerplexityBot, Claude-SearchBot. On-demand Fetches a specific URL in real time when an AI user asks a question that requires it. Synchronous with human input. Short bursts, often a single request per URL per query. Examples: ChatGPT-User, Perplexity-User, Claude-User. Figure 1: The three classes of AI crawler. Each class has distinct robots.txt semantics, log patterns, and business consequences when blocked. The class a crawler belongs to determines three things: the robots.txt directive that controls it, the log pattern you'll see, and whether blocking it has business consequences. Blocking a training crawler stops your content entering training data: which some sites want. Blocking an on-demand fetcher stops live AI users from ever seeing your page cited, which very few sites actually want. The Ten AI Crawlers Ordered by operator, grouped for quick reference. Each card has the exact user-agent string, the operator, purpose class, verification method, and a one-line Nginx log grep you can run right now. 01 GPTBot OpenAI · Training User-agent Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot) Purpose Retrieves publicly accessible content for OpenAI's foundation model training corpus. Does not contribute to ChatGPT's real-time search responses. Verification OpenAI publishes an IP range JSON at openai.com/gptbot.json. Match the requesting IP against that file, or use reverse DNS: legitimate GPTBot traffic resolves to a subdomain under openai.com. Log grep grep 'GPTBot' /var/log/nginx/access.log | wc -l robots.txt User-agent: GPTBot followed by Disallow: / blocks all GPTBot crawling. Honoured by OpenAI as documented policy. 02 ChatGPT-User OpenAI · On-demand User-agent Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ChatGPT-User/1.0; +https://openai.com/bot) Purpose Fetches a specific URL when a ChatGPT user clicks a link, asks about a specific page, or uses browsing to answer a question. Synchronous with user input. Verification Same IP range file as GPTBot. The distinguishing signal in logs is the user-agent string and the request pattern: single-URL fetches rather than site-wide crawls. Log grep grep 'ChatGPT-User' /var/log/nginx/access.log | awk '' | sort | uniq -c | sort -rn Output shows which specific URLs ChatGPT users are asking about in real time. 03 OAI-SearchBot OpenAI · Retrieval User-agent Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot) Purpose Crawls and indexes content for ChatGPT Search: the retrieval layer behind ChatGPT's answer generation when web grounding is enabled. Distinct from training and from on-demand browsing. Verification Same openai.com/searchbot.json IP range file. Note that this is a separate endpoint from GPTBot's: they publish distinct ranges. Why it matters OAI-SearchBot access is a prerequisite for appearing in ChatGPT Search results. Blocking it blocks citation in the search surface: which is distinct from blocking training data via GPTBot. 04 PerplexityBot Perplexity · Retrieval User-agent Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot) Purpose Background retrieval crawler. Indexes content for Perplexity's answer-generation retrieval layer. Scheduled revisits. Verification Perplexity publishes IP ranges at their documented endpoint. Reverse DNS on a legitimate PerplexityBot request resolves to a perplexity.ai subdomain. Log grep grep 'PerplexityBot' /var/log/nginx/access.log | awk '' | sort -u | head Lists unique IPs seen: useful for spot-checking against the published IP ranges. 05 Perplexity-User Perplexity · On-demand User-agent Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Perplexity-User/1.0; +https://perplexity.ai/perplexity-user) Purpose On-demand fetcher triggered when a Perplexity user asks a question that requires retrieving a specific URL in real time. The most common AI crawler source of "user just asked about my page" traffic in 2026. Log signature Single-URL requests, often correlated in time with spikes in referral traffic from perplexity.ai. If you see Perplexity-User hit a specific URL and traffic from perplexity.ai rise within the same hour, the page has just been cited in a live answer. 06 ClaudeBot Anthropic · Training User-agent Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com) Purpose Retrieves content for Claude model training. Analogous to GPTBot's role at OpenAI: training corpus construction, not live retrieval. Verification Anthropic publishes an IP range file; reverse DNS resolves to anthropic.com subdomains. The contact email in the user-agent, claudebot@anthropic.com, is a unique identifier absent from spoof user-agents. robots.txt User-agent: ClaudeBot followed by Disallow: / blocks training crawl. Does not affect Claude-User or Claude-SearchBot. 07 Claude-User Anthropic · On-demand User-agent Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-User/1.0; +https://www.anthropic.com/claude-user) Purpose On-demand fetcher triggered when a Claude user asks about a specific URL, or when Claude uses browsing in a conversation. Real-time, synchronous with user input. Log signature Single-URL fetches with no follow-up crawl. If you see Claude-User hit a page with no preceding or following requests from the same operator, a user is almost certainly asking Claude about that specific page right now. 08 Claude-SearchBot Anthropic · Retrieval User-agent Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-SearchBot/1.0; +https://www.anthropic.com/claude-searchbot) Purpose Retrieval crawler for Claude's web search and research tools. Scheduled crawls, populates the retrieval index consulted when Claude uses web search during a conversation. Why it matters The three Anthropic crawlers correspond to three distinct functions. Blocking ClaudeBot preserves the option to opt out of training; keeping Claude-SearchBot and Claude-User allowed keeps the site visible to Claude's live users. Conflating them blocks too much or too little. 09 Google-Extended Google · Training User-agent No distinct user-agent. Google-Extended is a robots.txt product token, not a separate crawler: Googlebot handles the actual fetching, and the token tells Google whether to use the content for Gemini and Vertex AI training. Purpose Controls whether content crawled by Googlebot is also used to train Gemini and Vertex AI generative models. The crawler itself is still Googlebot; Google-Extended is the opt-out mechanism. robots.txt User-agent: Google-Extended Disallow: / Blocks training use without affecting Google Search indexation. This is the only way to opt out of Gemini training while remaining in Google Search. Log grep Not applicable, no distinct user-agent appears in logs. Opt-out works through robots.txt compliance, not IP-level blocking. 10 Amazonbot Amazon · Training & retrieval User-agent Mozilla/5.0 (Linux; ... ) AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot) Purpose Amazon's general-purpose crawler, used for Alexa content sourcing and increasingly for Amazon's generative AI products. Crawls are scheduled rather than on-demand. Verification Reverse DNS resolves to crawl.amazonbot.amazon subdomains. Amazon publishes IP ranges in their developer documentation. Log grep grep -iE 'Amazonbot|AlexaBot' /var/log/nginx/access.log | wc -l The Ten at a Glance One reference table. Bookmark this if you want a quick lookup for what each user-agent string means and whether blocking it has consequences. # Crawler Operator Class If you block it 01 GPTBot OpenAI Training Content excluded from GPT training corpora 02 ChatGPT-User OpenAI On-demand ChatGPT users asking about your page see no content 03 OAI-SearchBot OpenAI Retrieval Not cited in ChatGPT Search 04 PerplexityBot Perplexity Retrieval Not indexed for Perplexity answers 05 Perplexity-User Perplexity On-demand Perplexity users querying your URL see nothing 06 ClaudeBot Anthropic Training Content excluded from Claude training corpora 07 Claude-User Anthropic On-demand Claude users can't retrieve your pages mid-conversation 08 Claude-SearchBot Anthropic Retrieval Not cited in Claude web search responses 09 Google-Extended Google Training Opt-out of Gemini and Vertex AI training 10 Amazonbot Amazon Training Excluded from Amazon AI products and Alexa Four Log Queries That Actually Tell You Something Four Nginx one-liners that turn raw access logs into usable AI crawler intelligence. Each assumes the default Nginx combined log format at /var/log/nginx/access.log: adjust the path if yours differs.After confirming AI crawler visits via logs, the next diagnostic is a GEO audit: the ten GEO audit checks identify which structural failures are most likely to suppress citation despite confirmed crawling. 1. Count all AI crawler requests in the last 24 hours. grep -iE 'GPTBot|ChatGPT-User|OAI-SearchBot|PerplexityBot|Perplexity-User|ClaudeBot|Claude-User|Claude-SearchBot|Amazonbot' /var/log/nginx/access.log | awk -v d="$(date -d '24 hours ago' '+%d/%b/%Y:%H')" '$4 > "["d' | wc -l 2. Which pages are on-demand fetchers hitting right now? This one tells you which URLs are being asked about in live AI conversations. grep -iE 'ChatGPT-User|Perplexity-User|Claude-User' /var/log/nginx/access.log | awk '' | sort | uniq -c | sort -rn | head -20 3. Which AI crawlers visit most often? Distribution across operators over the whole log file. grep -iE 'GPTBot|ChatGPT-User|OAI-SearchBot|PerplexityBot|Perplexity-User|ClaudeBot|Claude-User|Claude-SearchBot|Amazonbot' /var/log/nginx/access.log | grep -oiE 'GPTBot|ChatGPT-User|OAI-SearchBot|PerplexityBot|Perplexity-User|ClaudeBot|Claude-User|Claude-SearchBot|Amazonbot' | sort | uniq -c | sort -rn 4. What is the 4xx/5xx rate for AI crawlers? A high error rate for a specific crawler means it's hitting blocked pages, broken redirects, or pages behind auth. Every error is content that will never reach the retrieval index. grep -iE 'GPTBot|PerplexityBot|ClaudeBot|Claude-SearchBot|OAI-SearchBot' /var/log/nginx/access.log | awk '' | sort | uniq -c | sort -rn On user-agent spoofing. User-agent strings are trivial to fake. A meaningful fraction of traffic claiming to be GPTBot or PerplexityBot is not, it's scrapers and competitive intelligence tools using the AI crawler user-agent to avoid being blocked. For any count above low hundreds per day, verify a sample of IPs against the operator's published range file before acting on the numbers. What to Put in robots.txt Three common policies. Pick the one that matches your posture, not whichever one sounds most cautious by default.Crawl policy changes affect citation indirectly and with a lag: a citation measurement run before and after any robots.txt change is the only reliable way to quantify the effect. Policy A — allow everything. Default. Suitable for content that benefits from being in both training data and live AI answers. This is the right policy for most publishers who want to be cited. # robots.txt: allow all AI crawlers (default) # No AI-specific directives needed. Policy B — opt out of training, allow retrieval and on-demand. Keeps the site visible to live AI users and in AI search results, but excludes content from model training corpora. The compromise most publishers actually want. User-agent: GPTBot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: Amazonbot Disallow: / Policy C — block all AI. Nuclear option. Removes the site from every AI retrieval and training pipeline. Suitable for content with licensing restrictions or sensitivity that makes AI exposure unacceptable. The side-effect is total invisibility in AI-generated answers. User-agent: GPTBot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: OAI-SearchBot Disallow: / User-agent: PerplexityBot Disallow: / User-agent: Perplexity-User Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Claude-User Disallow: / User-agent: Claude-SearchBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: Amazonbot Disallow: / The GEO Lab runs Policy A. The whole point is to be cited. Quick reference for the trade-offs: Policy Training Retrieval On-demand Best for A — Allow all ✓ ✓ ✓ Publishers who want maximum citation potential B — No training ✗ ✓ ✓ Sites that want live AI visibility without training exposure C — Block all ✗ ✗ ✗ Content with licensing restrictions or legal sensitivity Frequently Asked Questions What is the difference between GPTBot and ChatGPT-User? GPTBot is OpenAI's training crawler: it retrieves content for model training and pre-training corpus construction. ChatGPT-User is the on-demand fetcher that retrieves a specific URL when a ChatGPT user asks about it in a conversation. The two have different user-agent strings, different crawl patterns, and different robots.txt semantics. Blocking one does not block the other, and treating them as a single entity in log analysis leads to misdiagnosed crawl behaviour. How do I verify an AI crawler is real and not a spoof? Three methods, in order of reliability: reverse DNS lookup on the requesting IP should resolve to a subdomain of the operator (for example openai.com or anthropic.com), then forward DNS on that hostname should resolve back to the same IP; match the IP against the operator's published IP range JSON file; and check request headers for operator-specific signatures. Reverse-and-forward DNS verification is the standard Google uses and is the most reliable for operators that publish verification endpoints. What user-agent does Perplexity use? Perplexity uses two distinct user-agents. PerplexityBot is the background crawler that indexes content for Perplexity's retrieval system. Perplexity-User is the on-demand fetcher triggered when a user asks Perplexity a question that requires retrieving a specific URL in real time. Both contain the literal string "Perplexity" in the user-agent header, but their crawl behaviour differs substantially: PerplexityBot crawls at scheduled intervals, while Perplexity-User fetches within seconds of a user query. Do AI crawlers execute JavaScript? Most AI crawlers do not execute JavaScript at crawl time. GPTBot, PerplexityBot, and ClaudeBot retrieve the raw HTML source and parse it without running client-side scripts. A minority of on-demand fetchers, typically the per-query browsing agents, may render JavaScript, but their behaviour is inconsistent across operators and model versions. Content that relies on client-side rendering is effectively invisible to the majority of AI crawler traffic. How do I block an AI crawler without blocking legitimate traffic? The correct layer is robots.txt, using the crawler's documented user-agent string and a Disallow directive. Firewall-level blocks risk collateral damage if the crawler shares IP ranges with other services. User-agent matching at the web server layer is effective for crawlers that honour robots.txt and legal for crawlers that do not, but it still requires the exact documented user-agent string: a partial match will miss variants. Related 10 GEO Terms Every Marketer Needs to Know in 2026 10 Technical Audit Checks Every GEO Page Should Pass What Is Generative Engine Optimisation (GEO)? The GEO Stack: Five Layers of AI Visibility Version History Version 1.0 — 12 May 2026: Initial publication. Ten AI crawlers across four operators, three purpose classes, with verification methods and Nginx log queries. About the author: The GEO Lab founder Artur Ferreira has 20+ years of experience in SEO and organic growth strategy. He developed the GEO Stack framework and leads research into Generative Engine Optimisation methodologies. Connect on X/Twitter or LinkedIn. Have questions? Contact The GEO Lab --- ## 10 Content Formats Mapped to Query Intents URL: https://thegeolab.net/10-content-formats-query-intent/ 10 Content Formats Mapped to Query Intents By Artur Ferreira · The GEO Lab · Published: 16 June 2026 · Version 1.0 Ten structural templates, one query intent each — the practitioner's format-to-intent pattern language that sits under every well-cited page TL;DR An AI system deciding which source to cite first matches the shape of the user's query to the shape of the content. "What is X?" wants a definitional page. "How do I...?" wants an ordered how-to. "X vs Y" wants a comparison table. When the shapes align, the page gets cited. When they don't, a competitor with the right shape wins. Ten formats cover most of the intent space: definitional page, how-to guide, comparison table, listicle, troubleshooting registry, case study, glossary entry, reference/spec, explainer, and FAQ page. Each has a primary intent, a secondary intent it also serves, and a set of intents it will fail on. Key GEO Takeaway AI retrieval systems evaluate format before they evaluate content: a well-structured how-to page beats a better-written wall of prose for procedural queries every time. Matching format to query intent is the structural decision that determines whether a page is retrievable before any keyword or schema work begins. The AI Reads Shape Before It Reads Content I wrote a very good post on FAQPage schema. It had the right information, the right sources, the right structure. It got retrieved. It did not get cited — because the query that would have pulled it in was phrased as a how-to ("how do I fix the duplicate FAQPage error") and the page was structured as an explainer. The AI system fetched the page, looked for an ordered sequence of steps to quote, found prose paragraphs instead, and went to quote someone else who had the exact same information in a numbered list.Format optimisation is Layer 2 work in the GEO Stack — specifically the Extractability sub-layer that determines whether a retrieved passage is quotable, independent of whether the page ranked in traditional search. The information was the same. The shape was different. Shape won. This is the thing I find practitioners underestimate most consistently in 2026. There's a tendency to think of content quality as a single dimension — well-researched, well-written, well-structured content will be cited. It won't, if the structure is wrong for the query. AI systems have format preferences per intent, and those preferences are sharp enough to override content quality within the same topic. The ten formats below are the structural templates that fit the common intents cleanly. Each one is a pattern you can see in the HTML — headings, layout, schema — not a vibe-based judgement about writing style. Pick the format first. Write the content second. Reversing that order is how thoroughly-researched posts lose citations to shallower ones. Query Intents and the Formats That Serve Them Before the ten format cards, the matrix that drives everything else. Six common query intents, and the formats that win each one.Testing how well a format serves a specific intent type requires a structured probe set — the ten probe query types are the Lab's instrument for measuring format-to-citation relationships empirically. Definitional "What is X?" — Definitional page, glossary entry Procedural "How do I...?" — How-to guide, reference/spec Comparative "X vs Y" — Comparison table, listicle Commercial/category "Best X" — Listicle, comparison table Troubleshooting "Fix [error]" — Troubleshooting registry, FAQ page Evidence-seeking "Does X work?" — Case study, explainer Figure 1: Six common query intents and the content formats that win each one. Most formats win one primary intent and serve a secondary one — few formats cover more than two. The mapping is not one-to-one. Some formats — the listicle, for example — serve two intents well (comparative and commercial) because their shape accommodates both. Some intents — troubleshooting — are served by more than one format, but with different strengths. The format cards below specify each format's primary intent, its secondary use, and the intents where it fails. The Ten Formats Each format card has the structural pattern, primary and secondary intent, the schema type that reinforces it, and a failure mode to watch for.Each format has a different extractability profile — the ten extractability signals document which structural properties make a given format easier or harder for AI systems to quote cleanly. 01 Definitional page Article Pattern Opens with "X is Y" in the first sentence. Structured as: what it is, why it exists, how it works, related concepts. One canonical definition per page. Primary intent Definitional. "What is X?" queries where the user wants the term defined unambiguously. Example query "What is citation rate in AI search?" Failure mode Narrative opening. Any setup before the definition costs the page the extraction slot — AI systems looking for "X is Y" at the top find a story instead. 02 How-to guide HowTo Pattern Numbered steps with action-oriented verbs at the start of each. Code blocks or concrete examples where applicable. Result or verification step at the end. Primary intent Procedural. Any query starting with "how do I" or "how to". Example query "How do I check my robots.txt for AI crawler rules?" Failure mode Prose descriptions of steps instead of an ordered list. AI systems cannot extract a numbered procedure from a paragraph that describes the procedure in free text. 03 Comparison table Article Pattern Explicit table element comparing two or more named entities across consistent criteria. Row per attribute, column per entity. Short summary of when each choice wins. Primary intent Comparative. "X vs Y" or "X compared to Y" queries. Example query "GEO Stack vs Answer Engine Optimization — what's the difference?" Failure mode Prose comparison without a table. A well-written comparison in three paragraphs loses to an average comparison in a clean table, because the table is directly extractable as structured output. 04 Listicle ItemList Pattern Numbered list of discrete items, each with a consistent internal structure. Named entities prominent. Section-per-item format preserves extractability for each item individually. Primary intent Commercial/category. "Best X", "top Y", "top 10" queries. Example query "Best AI visibility monitoring tools in 2026" Failure mode Uneven item structure — some items with three sub-sections, others with one paragraph. AI systems extracting a list of items expect the same shape for each, and inconsistency forces them to skip items or flatten the list. 05 Troubleshooting registry Article Pattern Entries structured as symptom → root cause → fix. The exact error string or error message appears verbatim in the entry. Often organised as a registry of multiple entries on one page or one entry per page. Primary intent Troubleshooting. Queries that include a specific error string or describe a specific failure. Example query "Google Search Console 'Duplicate field: FAQPage' error — how to fix" Failure mode Paraphrasing the error message instead of quoting it verbatim. AI troubleshooting retrieval does string-level matching against the reported error before anything else — paraphrasing breaks that match. 06 Case study Article Pattern Structured around: context, problem, intervention, measurement, result. Named actors (company, team, named individual). Specific numbers for before and after. Timeline present. Primary intent Evidence-seeking. Queries asking whether a technique works, what results others got, or how to judge an approach. Example query "Has anyone measured citation rate improvement from adding schema?" Failure mode No numbers, no timeline, no named actors. A case study in name only — narrative reads well, produces no extractable evidence. AI systems evaluating evidence cite case studies with data, not case studies with stories. 07 Glossary entry DefinedTerm Pattern Short, focused definition (50–200 words) of a single term. Often part of a larger glossary collection. No narrative, no expansion — just the term and the canonical definition. Primary intent Definitional. Identical primary intent to the definitional page, but at smaller scale and higher density. Example query "Define C-SOV in GEO measurement" Failure mode Treating the glossary entry as a stub that redirects to a longer page. A glossary entry should be complete on its own — redirecting the reader to a different page to find the actual definition is how glossary entries stop being cited. 08 Reference/spec TechArticle Pattern Comprehensive, flat documentation of a system, API, schema, or protocol. Organised by component or endpoint rather than by narrative. Every item has a consistent structure — name, description, parameters, example, notes. Primary intent Procedural (secondary) and definitional (secondary). Queries that look up a specific parameter, flag, field, or configuration option. Example query "What does the user-agent parameter do in Perplexity API requests?" Failure mode Narrative reference. Trying to explain a reference document in prose rather than as a flat lookup. References need to be scan-and-extract, not read-from-start-to-finish. 09 Explainer Article Pattern Prose-heavy, narrative-driven content that walks through a concept with examples, analogies, and gradual buildup. Designed to be read linearly. Primary intent Evidence-seeking (secondary) and broad educational intent. Queries phrased as "why does X happen" or "what's the intuition behind Y". Example query "Why do AI systems cite some sources more than others?" Failure mode Using explainer format for definitional or procedural queries. An explainer is the right choice for a handful of intents — for most queries, a definitional page or how-to would win. Default to explainer format is the single most common format-to-intent mismatch. 10 FAQ page FAQPage Pattern Explicit question-answer pairs. Each question phrased as a user would ask it. Each answer self-contained and short. FAQPage schema directly mirrors the visible content. Primary intent Troubleshooting (secondary) and "common questions" queries. FAQ pages win when users are asking directly and concisely. Example query "Do AI crawlers execute JavaScript?" Failure mode Stretching the FAQ format into a primary content format. FAQ works as a secondary section on a longer page, or as a dedicated FAQ landing page for support use cases. Trying to make FAQ the primary structure of the entire site turns every topic into fragmented Q&A with no connective tissue. The Ten at a Glance Reference table. Use when you're about to write a page: identify the primary query intent first, then match to the format row. # Format Schema Primary intent Also serves 01 Definitional page Article Definitional Evidence-seeking 02 How-to guide HowTo Procedural Troubleshooting 03 Comparison table Article Comparative Commercial 04 Listicle ItemList Commercial Comparative 05 Troubleshooting registry Article Troubleshooting Procedural 06 Case study Article Evidence-seeking Comparative 07 Glossary entry DefinedTerm Definitional — 08 Reference/spec TechArticle Procedural Definitional 09 Explainer Article Evidence-seeking Definitional 10 FAQ page FAQPage Troubleshooting Definitional When to Combine and When to Split The temptation to combine formats — one page covering definitional, comparative, and procedural intent for the same topic — is the single most common structural mistake in content strategy. It produces pages that read as "thorough" to a human and as "unfocused" to an AI retrieval system.The citation evidence on format performance by intent type comes from Tier 1 vs Tier 2 experiments — the citation gap post shows how query type and format interact to produce citation rate differences of 60 percentage points or more. Three principles make the combine-vs-split decision tractable. First: one primary intent per page. A page can reference other intents through links, but the page itself needs to be unambiguous about what it is. A definitional page that also tries to be a how-to ends up failing both — the opening isn't declarative enough for the definitional intent, and the how-to steps are interrupted by definitional context. Second: pair formats only when they share structural shape. A listicle and a comparison table share shape — enumerated items with consistent attributes — and combine well. A definitional page and a troubleshooting registry do not — one is narrative-driven, the other is lookup-driven, and forcing them onto the same page breaks both patterns. Third: cross-link instead of combining. The right response to "this topic has three intents" is three pages, each in the right format for its intent, with explicit cross-links between them. The AI system then has three clean extraction targets instead of one confused one. Practitioners used to SEO often resist this because it feels like "thin content" — but thin content is content with nothing to say, not content that says one thing well. A working pattern applied on thegeolab.net: Topic Split into Cross-linked through Citation rate Definitional page + glossary entry + case study Definitional page links to glossary and case study; case study links back to definitional FAQPage duplicate error Troubleshooting registry entry + how-to guide + FAQ entry Registry entry links to how-to; how-to has short FAQ block at end GEO vs AEO vs LLM SEO Comparison table page + three separate definitional pages, one per term Each definitional page linked from the comparison row for that term On schema-format alignment. Every row in the table above aligns the schema type to the format. Article on definitional, HowTo on the how-to guide, FAQPage only where there is actual question-answer structure. Misaligned schema — FAQPage schema on a definitional page, HowTo on a troubleshooting registry — is worse than no schema. The AI system detects the mismatch between what the schema promises and what the content delivers, and discounts both. Frequently Asked Questions What is a content format in GEO? A content format is a structural template that organises information in a specific way — a definitional page, a how-to guide, a comparison table, a troubleshooting registry. The format is distinct from the topic. Two pages on the same topic can be structured in different formats, and each format serves a different set of query intents. Choosing the right format for the query intent is often the difference between a page that gets cited and one that doesn't. How do I know which content format matches a query intent? Match the structural shape of the query to the structural shape of the format. "What is X?" queries are definitional — they want a definitional page opening with "X is Y". "How do I..." queries are procedural — they want an ordered how-to with numbered steps. "X vs Y" queries are comparative — they want a side-by-side comparison table. When the query shape and format shape align, the AI system extracts a clean answer. When they don't, the page loses to a competitor whose format matches better. Can one page serve multiple query intents? Yes, but with diminishing returns. A page that tries to be a definitional page, a how-to, and a comparison all at once usually fails at all three — the structural patterns required by each format conflict. The better approach is separate pages for separate intents, cross-linked, with each page optimised for its primary format. Trying to cover three intents on one page is how "comprehensive" posts end up citation-less despite being topically thorough. What format works best for commercial or category queries? Commercial and category queries ("best X tools", "X vs Y alternatives", "top Y providers") favour list pages with named entities, brief descriptions, and explicit criteria. AI systems responding to category queries strongly prefer content that names competitors, structures them as a list, and gives comparable data points for each. Category queries almost never cite pure narrative content, regardless of quality. Does schema type need to match the content format? Yes. HowTo schema on a how-to page reinforces the extraction signal; HowTo schema on a narrative page creates a mismatch that weakens trust in both. FAQPage schema works on pages with clear question-answer structure and nowhere else. The rule is simple: the schema type should describe what the content actually is, not what you wish it were. Misaligned schema is worse than no schema. Related 10 Probe Queries We Use to Test AI Citation Behaviour 10 Extractability Signals That Predict AI Citation 10 GEO Terms Every Marketer Needs to Know in 2026 The GEO Stack: Five Layers of AI Visibility Version History Version 1.0 — 16 June 2026: Initial publication. Ten content formats mapped to six common query intents, with schema alignment and combine-vs-split principles. About the author: The GEO Lab founder Artur Ferreira has 20+ years of experience in SEO and organic growth strategy. He developed the GEO Stack framework and leads research into Generative Engine Optimisation methodologies. Connect on X/Twitter or LinkedIn. Have questions? Contact The GEO Lab --- ## 10 Extractability Signals That Predict AI Citation URL: https://thegeolab.net/10-extractability-signals-ai-citation/ 10 Extractability Signals That Predict AI Citation By Artur Ferreira · The GEO Lab · Published: 2 June 2026 · Version 1.0 Ten structural patterns that make content easier for AI systems to quote cleanly — the practitioner reference for Layer 2 of the GEO Stack TL;DR A page that an AI system retrieves but can't extract a clean answer from gets skipped in favour of one it can. Ten structural patterns correlate with high extractability: declarative opening, heading-as-question, short answer before long answer, paragraph length discipline, early source attribution, entity density, schema-content alignment, explicit numbers and units, single canonical answer per section, and machine-readable markup. Each signal is a pattern you can see in the HTML, not a content quality judgement. That's what makes them testable, fixable, and measurable. Key GEO Takeaway Being crawled is not the same as being cited: AI systems retrieve thousands of pages and quote a fraction of them. Extractability — the structural property that makes a passage quotable — is the gap between those two numbers, and it is measurable across ten specific signals before a single query is run. Retrieved but Not Cited The first time I saw a GPTBot access log entry on a page that had never been cited in a ChatGPT answer, I assumed the crawler just hadn't gotten around to indexing the content yet. Then I watched it happen eleven more times over the following weeks — same page, repeated fetches, zero citations. The page was in the retrieval index. It was being read. It wasn't being quoted.Extractability is Layer 2 of the GEO Stack — the layer between crawlability (Layer 1) and structural authority (Layer 3) that determines whether a retrieved page produces a quotable passage. That gap — between retrieval and citation — is what Layer 2 of the GEO Stack is named for. The AI system found the page. It just couldn't find a clean sentence to quote from it. So it quoted somebody else's page, which had the same information but structured in a more extractable shape. The uncomfortable version of this: the page I lost the citation on was better researched than the page that won it. Extractability is not a quality signal. It's a shape signal. The AI can't tell which page is better. It can only tell which page gives it a quotable answer without having to synthesise one from disconnected fragments. The ten signals below are what separates an extractable page from an unextractable one. All of them are patterns in the HTML, which means all of them are measurable, and all of them are things you can change. Where Extraction Happens Extraction runs in three sub-layers inside Layer 2 of the GEO Stack. Knowing which sub-layer a signal targets tells you which kind of fix to apply.Each sub-layer has specific, testable failure modes — the ten GEO audit checks operationalise extraction diagnostics into pass/fail tests that can be run programmatically before deployment. Sub-layer 1 Answer location Signals 01, 02, 03 Sub-layer 2 Answer isolation Signals 04, 05, 09 Sub-layer 3 Answer reinforcement Signals 06, 07, 08, 10 Figure 1: The three sub-layers of extraction. Answer location decides whether the AI finds the answer; answer isolation decides whether it can pull a clean chunk; answer reinforcement decides whether it trusts what it pulled enough to quote. A page can fail at any sub-layer. A page with a declarative opening (good location signal) but five-sentence paragraphs (bad isolation signal) gets found but not cleanly chunked. A page with clean chunks but no source attribution (bad reinforcement signal) gets chunked but quoted as "according to a source" without the link. Citation requires all three sub-layers to clear. The Ten Extractability Signals Ordered by observed impact on extraction behaviour in the Lab's experiments. The first four are the ones that swing the outcome. The last six are structural polish — smaller effects individually, substantial in combination.Which signals matter most depends on the query type being probed — the ten content formats mapped to query intent shows which format choices naturally improve signal scores for different query types. 01 Declarative opening sentence High What it is The first sentence of content states a definitional fact, typically in "X is Y" form, within the first 200 characters. Why it works AI systems extracting a clean answer look near the top of the content first. A declarative opening is a sentence already shaped like an answer. A narrative hook is not. Examples Extractable "Citation rate is the proportion of AI query iterations in which a given URL appears as a named source link." Not extractable "Let's talk about citation rate. It's one of those metrics that everyone agrees matters, but nobody quite defines the same way." 02 Heading phrased as the question High What it is Section headings use the same phrasing a user would type into an AI system to ask about the topic the section covers. Why it works Heading-level matches between user queries and content give the retrieval system a direct query-to-section pointer. The section under the matching heading is then the natural extraction target. Examples Extractable "How do I block GPTBot in robots.txt?" Not extractable "Bot management considerations" 03 Short answer before long answer High What it is The first paragraph under a heading gives the complete answer in one or two sentences. Subsequent paragraphs expand, nuance, or justify — but the answer itself is extractable from the opening. Why it works AI systems generating a concise response prefer a self-contained opening paragraph they can quote without modification. Long answers that build up to a conclusion leave the AI either quoting the build-up (wrong) or synthesising a summary (which it then doesn't cite to the source). Pattern Short answer → long answer. Not the other way around. 04 Paragraph length discipline High What it is Body paragraphs are two to four sentences long. Single-sentence paragraphs are used sparingly for emphasis. Five-plus-sentence paragraphs are avoided. Why it works AI systems chunk content at paragraph boundaries. Paragraphs longer than five sentences produce chunks too large to fit into a generated response without truncation. Truncated chunks are either not used or used with a "..." that reduces citation trust. Target range Two to four sentences per paragraph for body content. Longer is permissible only when the content is explicitly a single connected argument that can't be split. 05 Single canonical answer per section Medium What it is Each section of the page answers exactly one question. Sections do not contain multiple competing answers, alternative positions, or conditional phrasings that the AI would have to choose between. Why it works AI systems extracting an answer want a section where the answer is unambiguous. A section with three conditional answers ("it depends on...") forces the AI to either quote one condition without context or synthesise across them. Both reduce citation likelihood. Pattern One question, one section, one canonical answer. Nuances and exceptions go in their own sections — not inside the primary answer section. 06 Early source attribution Medium What it is Claims made in the first few paragraphs of a page are attributed to named sources — publications, studies, or specific experts — rather than left as anonymous assertions. Why it works AI systems trained on academic and journalistic content strongly prefer to cite sources that themselves cite sources. A page with "according to [X study]" near the top reads as authoritative; a page with "it's widely known that..." reads as unsourced. Pattern Named source within the first 400 characters. Hyperlink to the source. Year of the source where applicable. 07 Entity density Medium What it is The content contains multiple named entities — specific products, people, companies, frameworks — within each section, not just one general topic mentioned throughout. Why it works AI retrieval systems use entity co-occurrence as a topical signal. A section that names five entities in context carries more retrieval weight than a section that names one entity repeatedly. Entity density also sharpens the match between user queries and content. Pattern At least three distinct named entities per major section, with at least one entity appearing in the section heading or opening sentence. 08 Explicit numbers and units Medium What it is Quantitative claims in the content state specific numbers with units, rather than relying on qualitative terms like "significant", "many", or "a large percentage". Why it works The research literature on GEO shows that content with embedded statistics is cited more often than content without. AI systems responding to questions that ask for numbers need content with numbers in it — qualitative content forces them to either skip the question or generate a number without citation. Examples Extractable "Citation rate shifted by 11 percentage points between the two conditions." Not extractable "Citation rate shifted significantly between the two conditions." 09 Schema-content alignment Supporting What it is Structured data declarations on the page accurately describe the content. FAQPage schema only on pages with clear Q&A structure. HowTo schema only on pages with ordered steps. Article schema only on pages with a single coherent article. Why it works Schema is an extractability amplifier when aligned and a liability when not. FAQPage schema on narrative prose creates a mismatch that AI systems detect, which reduces trust in both the schema and the surrounding content. Common failure FAQPage schema generated by a plugin on pages without a real FAQ section. The signal is worse than no schema, because the AI expected structured Q&A and got something else. 10 Machine-readable markup Supporting What it is Lists are marked up as