Technical SEO Field Reference, Question 14 of 66. The duplicate content question, checked against what Google currently documents.
Large websites should handle duplicate content at the source: stop unnecessary URL variants from being generated, then align redirects, canonical tags, internal links and sitemaps on one preferred URL. Canonical tags help with consolidation, but they do not scale well if the site keeps producing new duplicate content underneath them (Google Search Central, as of September 2026).
On a large website, duplicate content is rarely one page copied twice. It usually comes from the machinery underneath the site: filters, tracking parameters, session IDs, print versions, protocol variants, old URL structures or several routes to the same product. That is why “add a canonical tag” is only part of the answer. The tag may help Google consolidate the URLs you already have, but it does not stop the platform from producing thousands more.
Why duplicate content matters on large sites
Small sites rarely have a serious duplicate content problem. Large sites almost always do, and it usually is not the kind interviewers are picturing when they ask this question. Most candidates jump straight to “add canonical tags,” which is correct but incomplete – it treats the symptom on one URL at a time instead of the structural reasons a large site keeps generating duplicates faster than canonicals can consolidate them.
The real cost of duplicate content on a large site is not a penalty. It is wasted crawl budget spent on URLs you did not want Google to prioritise, diluted signals across near-identical pages, and muddled reporting in Search Console.
What Google says about duplicate content
Google groups duplicate or near-duplicate pages and chooses one representative URL, the canonical, to show in search results. Its canonicalisation documentation makes clear that some duplication is normal. Sorting and filtering, regional variants, HTTP and HTTPS versions, and accidental test pages can all create alternative URLs.
Most duplicate content is not a spam violation. Google’s own guidance frames it as a normal, non-deceptive byproduct of how sites are built, handled through canonicalisation rather than punishment.
When you want to nominate a preferred URL, Google treats redirects and rel="canonical" as strong signals, while sitemap inclusion is weaker. The signals can reinforce one another. This is the same canonical hint mechanism covered in Question 12 of this series, so I will not repeat the full explanation here.
The interview answer
On a large website, duplicate content is a structural problem, not a per-page one, so the fix has to be structural too. Remove unnecessary URL variants where possible, redirect retired versions, use consistent self-referencing canonical tags, link internally to the preferred URLs and keep only those URLs in the XML sitemap. Canonical tags help with consolidation, but they do not scale if the site keeps generating new duplicate content underneath them.
A real-world example
A marketplace site had three duplicate-producing patterns running at once: session-ID parameters appended by the platform, a legacy www subdomain still resolving instead of 301-redirecting, and category pages reachable through two different URL structures left over from a prior site restructure.
Canonical tags were present and technically correct on most pages, but log files showed Googlebot still spending meaningful crawl activity on the parameterised and www variants. Old backlinks pointed to the legacy URLs, internal links included session parameters, and an outdated sitemap kept submitting the old categories. The canonical was asking for one URL while the rest of the site kept pointing somewhere else.
The fix was not more canonical tags. It was architectural: 301-redirect the www subdomain at server level, strip session IDs from internally linked URLs, regenerate the sitemap to include only the canonical category structure, and let the canonical tags do the consolidation work they were already set up to do once the conflicting signals stopped fighting them.
This connects to why pages end up Crawled – currently not indexed in Search Console. When Google finds duplicate content across multiple URLs, it picks one canonical and may drop the others from the index entirely – not as a penalty, but as a consolidation decision.
How duplicate content consolidation and elimination work
I split the work into two tiers: consolidation and elimination. That is my operational framing, not terminology Google uses in its documentation.
Consolidation handles duplicate content that needs to remain accessible. Currency variants, regional pages, print-friendly versions – these are legitimate URLs where only one should be indexed per market. Redirects, canonical tags, consistent internal links and sitemap hygiene point Google towards the preferred version.
Elimination stops unnecessary duplicate content from being created or discovered in the first place. That means removing session IDs from URLs, controlling faceted navigation output, correcting URL generation in the CMS, or fixing internal links at their source rather than tagging around them after the fact.
Elimination usually scales better. Every duplicate that remains needs to be crawled or otherwise discovered before Google can recognise and consolidate it. Google’s own ecommerce URL guidance warns that alternative URLs can lead to repeated crawling of the same content and unnecessary server load.
The common duplicate content sources on large sites, roughly by frequency: URL parameters from filters, sorts, and tracking codes; protocol and subdomain variants (HTTP vs HTTPS, www vs non-www); pagination with inconsistent self-canonicals; content syndication where the syndicated copy outranks the original; and CMS-generated alternative paths to the same content. Each source needs a different fix. Parameters need stripping or consistent handling. Protocol variants need server-level redirects. Pagination needs self-referencing canonicals per page. Syndication needs cross-domain canonicals or attribution agreements.
The measurement surface for duplicate content is Google Search Console’s Pages report, filtered by “Duplicate without user-selected canonical” and “Duplicate, Google chose different canonical than user.” If either count is growing, your consolidation signals are not winning.
The interview trap
The obvious trap is stopping at rel="canonical". On a large site, that manages the duplicate content already in circulation while filters, parameters and legacy routes continue creating more. A complete answer addresses the generator, not just the tag.
The other trap is calling ordinary duplicate content a penalty. Google normally handles it through clustering and canonical selection. The cost is more likely to show up as wasted crawl activity, muddled reporting, or signals spread across several versions of the same page – not a manual action or algorithmic punishment.
Takeaway
The wrong question is “Which pages need a canonical tag?” The better question is “What in my site’s architecture keeps generating duplicate content, and can I stop it at the source instead of tagging around it forever?” Start with the generator, not the tag.
Have questions about this topic? Contact The GEO Lab · Return to homepage
Have questions about this topic? Contact The GEO Lab · Return to homepage