Google AI Overviews vs Featured Snippets: Where Answers Disagree

Research review · Hu et al. · Google consumer search · Historical audit · 7 min read

Seeing your page cited in an AI Overview is encouraging. I’d still read the answer before putting it in the wins column.

A summary might name the right business but leave out a condition attached to its pricing or advice. A visibility report can record the link and miss that problem altogether. That is the AI Overviews vs Featured Snippets issue in practice: a page can be cited and misrepresented at the same time.

That’s the question this paper brings into focus for me. When Google offers two prominent answers to the same query, do they agree? And when they don’t, what should we check before deciding whether our content has been represented properly?

What the researchers measured

Desheng Hu and colleagues collected English-language Google results using logged-out Chrome, with the main collection set to a fixed US location in New York. They saved the results pages and manually assessed answer consistency and other quality measures.

Their paper compares AI Overviews vs Featured Snippets. These are different features, so the comparison asks whether the answers agree, rather than assuming one is a reference answer for the other.

The dataset contained 1,508 queries. Both features appeared visibly together for 322 of them. Those 322 pairs are the denominator for the disagreement result.

The abstract rounds the full-answer figure to 33%. I’ve used the more precise figure from the results section.

The safeguard measure concerns cues such as advice to consult a health professional. An answer without that cue isn’t automatically incorrect. Likewise, the confirmation-bias result belongs to the study’s particular test; it doesn’t rule out bias in every search setting.

The Zenodo record, arXiv paper and ICWSM conference publication belong to the same work family. They give us several ways to access one study, not several independent replications.

What AI Overviews vs Featured Snippets disagreement tells us

An inconsistent pair gives us something to investigate. We still need to assess the answers before saying which one is wrong.

That sounds obvious, but it’s easy to lose the distinction when reporting a percentage. “32.3% inconsistent” can quickly become “Google gets a third of its answers wrong”. The paper doesn’t establish that claim.

For a business page, I’d compare the captured wording with the relevant specification, offer or service terms. Specialist guidance needs a reference and reviewer suited to the subject.

I’d also save the source page as it stood at the time. If someone reviews the mismatch after the page has changed, they may be comparing the captured answer with information that wasn’t there during collection.

Check the highlighted words too

The highlighted-text result deserves its own check: the reported disagreement rate for AI Overviews vs Featured Snippets was 40.7%, compared with 32.3% for full answers.

I’d read the highlighted passage by itself, then read it again with the surrounding answer. That lets me check what someone scanning the page would encounter, as well as what the fuller response says.

For a commercial audit, I’d look for qualifications such as “up to”, “after approval” or “for eligible customers”. Removing one can change what an offer appears to promise. These are examples I’d use in an audit, not commercial cases tested by Hu and colleagues.

When writing my own pages, I’d keep a qualification close to the statement it applies to. That’s an editorial precaution. This study didn’t test whether moving a qualification improves AI extraction.

The retrieval and extraction pipeline helps explain why I’d investigate the wording as a separate part of the audit.

What this means for a GEO audit

My reading of the paper is that an AI visibility report needs room for the answer itself.

A citation records visible attribution. It doesn’t, on its own, establish which source supplied each part of an answer or whether the wording faithfully reflects the linked page. That’s why I’d review representation alongside the citation count.

The distinction is useful when a business needs to decide what to investigate. A missing link and a linked answer that misstates an offer call for different checks.

The authors discuss risks around health information. For GEO work, I’d take a narrower practical step: add a representation check to the report. The study supports investigating disagreement; the audit procedure below is my proposal for applying that lesson. A citation that misrepresents a condition is worse than no citation at all — and a retrieval event without a visible citation is a different problem entirely.

How I’d check AI Overviews vs Featured Snippets on a business website

I’d start with a small set of queries where misleading wording could change a customer’s decision. Prices, eligibility rules and service boundaries would be sensible places to begin.

This could become an additional part of a GEO audit. It isn’t a replication already completed by The GEO Lab.

Save each answer separately

When both features appear, record the full AI Overview and Featured Snippet separately. Keep the highlighted text and visible links with each capture.

Record the query, date, location and account conditions too. If one feature is absent, log it as absent. A result from another query won’t fill that gap.

I’d keep the features separate in the report, just as I would when comparing AI Mode with AI Overviews.

Give each outcome its own field

These fields let you describe a result more usefully than “visible” or “not visible”.

For example, a business could be named and linked, while its service area is described incorrectly. The mention and citation checks would pass. The representation check would flag the wording for review.

That’s an illustrative example, not a result from this study.

Repeat a mismatch before changing the page

I’d retain a single mismatch as an observation, then repeat the query under recorded conditions before treating it as persistent.

Choose the repetition rule before collecting results. Keep failed or unresolved captures in the log so the final percentage has an honest denominator. As with AI citation by query type, an indeterminate result needs to remain identifiable.

If the same condition keeps disappearing, you have a specific issue to investigate. Any later content change should be tested on its own; this paper doesn’t tell us which edit will fix it.

What needs a fresh test

The main collection took place on 8–9 April 2025. It covered English-language baby-care and pregnancy queries from a fixed US location. The paper also describes additional location checks, but those don’t turn it into a present-day, worldwide estimate.

I’d need fresh observations before attaching a rate to another topic, country or date.

A replication of the AI Overviews vs Featured Snippets comparison should fix its query set and define disagreement before coding. Two answers can differ in detail while remaining consistent. The coding needs to distinguish that from a contradiction.

For this article, the historical result justifies checking answer wording. A current audit would tell us how often that check finds a problem on the website and queries being tested.

Frequently asked questions