Technical SEO Field Reference, Question 01 of 66. The interview answer, checked against what Google currently documents.
Crawl demand is how much Google wants to crawl a site’s URLs, driven by perceived inventory, popularity, and staleness. Crawl budget is the set of URLs Googlebot can and wants to crawl, determined by crawl demand together with the site’s crawl capacity limit. Google rewrote this documentation on 22 July 2026, so most circulating interview answers describe the older wording (Google Search Central, as of August 2026).
This sounds like a simple definition question. It is not. A good answer shows whether you understand why Google crawls some URLs constantly and barely touches others. And since July 2026, it also shows whether your answer is current or ten years old.
Why it matters
Google’s own guidance scopes crawl budget to large sites. Yet r/SEO fills weekly with small-site owners watching pages sit uncrawled, manually submitting URLs and hoping for the best. barryhunter, a Diamond Product Expert in Google’s own Search Central community, calls the documentation “somewhat unhelpfully” narrow here: in his observation, smaller sites hit the same wall because per-page HTTP requests for JavaScript and CSS consume crawl capacity, not page count alone.
Knowing where the documentation ends and observed behaviour begins is exactly what this question tests. That line moved on 22 July 2026.
What Google actually says
Google defines crawl budget using two components (as of August 2026):
Crawl capacity limit. The total time a server spends holding connections open for Google, factoring in both the number of parallel connections and their duration. It rises when the site responds fast and healthy, falls on slowdowns and server errors.
Crawl demand. How much Google wants to crawl the site’s URLs, influenced by perceived inventory, popularity, and staleness.
Taken together: crawl budget is the set of URLs Google can and wants to crawl. Even when the capacity limit is not reached, low demand means less crawling.
| Crawl capacity limit | Crawl demand | Crawl budget | |
|---|---|---|---|
| What it is | What Google can crawl | What Google wants to crawl | What Google can and wants to crawl |
| Set by | Server health, response speed, error rate | Perceived inventory, popularity, staleness | Capacity limit + demand |
| Moves up when | Site responds fast and stays healthy | Content is popular, fresh, or newly discovered | Either input rises |
| Moves down when | Site slows or returns 5xx/429 | Content is stale or low interest | Either input falls |
| More URLs raises it? | No | No | No |
| Where you diagnose it | Crawl Stats report + server logs | Crawl patterns, discovery and refresh rate | Crawl Stats report + server logs |
| Common mistake | Assuming a faster server creates demand | Assuming crawlable means worth crawling | Treating it as a fixed allowance |
Two things the July 2026 rewrite added that older answers miss:
- Every site starts with the same default, conservative crawl capacity limit. Google raises it only when demand exists and the site stays healthy. There is no earned baseline you carry around.
- Crawl capacity is shared across all of Google’s crawlers, while each crawler has its own demand. Heavy crawling by one Google crawler reduces the capacity available to the others.
The interview answer
Crawl demand is Google’s interest in crawling your URLs. Crawl budget is the crawling Googlebot is willing and able to do, set by that demand plus your site’s crawl capacity limit. A site can expose millions of crawlable URLs and still see little crawling, because demand, not URL count, is the constraint. And since the July 2026 documentation update, we know every site starts from the same conservative capacity default and that capacity is shared across all Google crawlers.
Real-world example
An e-commerce site has 2 million URLs. Only 100,000 are valuable product and category pages. Faceted navigation generates the other 1.9 million parameter combinations such as ?colour=black&size=10&sort=price.
Googlebot spends significant activity on parameter URLs while important product pages refresh slowly. This is not SEO folklore: Google’s faceted navigation documentation describes exactly this failure, parameter-based facets generating infinite URL spaces that cause overcrawling and slower discovery of useful URLs.
The wrong move is declaring “we need more crawl budget.” The right move is the logs:
- Which URL patterns is Googlebot actually requesting, and how often?
- Are important URLs crawled less than expected?
- Do faceted URLs create an effectively unlimited crawl space?
- Is the server slow, or returning 5xx/429?
- Are other Google crawlers, or aggressive third-party and AI crawlers, consuming the shared capacity and server headroom?
Then: control the crawl space, improve internal discovery, keep sitemaps clean, and improve server performance where the logs say it is the constraint.
The technical detail
Demand and capacity interact asymmetrically. Capacity is a ceiling your server behaviour moves; demand is a floor Google’s perception of your content moves. Fixing a slow server raises the ceiling but does nothing for a site full of URLs Google has no interest in. Pruning worthless URL inventory raises effective demand per valuable URL without touching the server.
The measurement surface is GSC’s Crawl Stats report (requests per day, response times, response codes) plus server access logs, which show per-URL-pattern reality that GSC aggregates away.
The interview trap
Treating crawl budget as a fixed allowance Google grants every site. It is not an allowance; it is the intersection of what your server can take and what Google currently wants. And for most small and medium sites, crawl budget is not something to optimise at all: Google’s guidance targets large sites and sites that mass-generate URLs.
A second trap hides inside the first: “more crawling means better rankings.” Google’s own Myths and facts about crawling page rules this false: crawling is necessary for a page to appear in results, but it is not a ranking signal. Same page: the crawl-delay robots.txt rule is not processed by Google’s crawlers, so citing it as a crawl-budget lever fails twice.
The 2026 version of the trap: reciting the pre-rewrite answer confidently. An interviewer who read the July 2026 update will notice.
Takeaway
The question is never “how do we get more crawl budget.” It is “is Google spending its crawling on the URLs that matter.”
Have questions about this topic? Contact The GEO Lab · Return to homepage
Have questions about this topic? Contact The GEO Lab · Return to homepage