Research review · Allaham and Diakopoulos · Consumer interfaces · Detector-based audit
TL;DR
An audit of AI-generated sources in AI search flagged 3,056 of 19,154 successfully scraped cited URLs as “Likely AI” or “Highly Likely AI”, about 16%. That is a detector result for the analysed text, not verified authorship of every flagged page. The unanalysed URLs and detector-validation limits constrain any prevalence claim. The finding also does not establish that the flagged sources were inaccurate or that engines prefer AI-written content.
“AI search cites AI-written content” is a headline I would open. I would want to know how the authors established the writing process before repeating the number.
A detector can classify a passage. An editorial record can document how it was produced. A source check can assess whether its claims are supported.
Those are different pieces of evidence. A useful review keeps them visible rather than letting one label answer every question.
This paper gives us a citation audit worth inspecting, with a more limited conclusion than “16% of everything AI cites was written by AI”.
What the audit counted about AI-generated sources in AI search
Mowafak Allaham and Nicholas Diakopoulos submitted 712 queries to the consumer interfaces of ChatGPT, Copilot, Gemini and Perplexity, then examined cited URLs using Pangram. The paper reviewed here is arXiv v1, dated 22 May 2026.
The final row is a subset of the scraped URLs. It is not a separate collection to add to the total.
For the headline calculation, 3,056 divided by 19,154 is approximately 16%. The unit is a unique cited URL, not a response, user or repeated citation event.
That unit determines what the figure can describe.
A detector label needs to remain a detector label
I would report a flagged page as flagged unless there is additional evidence about its production.
That wording allows the detector result to be used without giving it more authority than the observation supports. It also makes the finding easier to revisit if the classifier or underlying text changes.
A detector result does not establish who wrote a sentence, whether an editor checked it or whether the content is true.
For example, imagine a page explaining a product’s warranty. It could contain AI-assisted prose while accurately reflecting the warranty terms. Another page could be entirely human-written and describe those terms incorrectly.
These are hypothetical cases. Their point is to keep production method and claim accuracy as separate review questions.
The missing URLs belong beside the percentage
The unanalysed portion is approximately 27.1% of collected URLs. The paper describes format and access barriers, including non-text content.
That gap does not make the measured subset useless. It does limit the population the result directly describes.
If the pages that can be scraped differ from those that cannot, a percentage from the first group may not carry over to the second. We need evidence before treating the missing group as equivalent.
Do not silently assign the missing pages a label
I would not count an inaccessible page as human-written, AI-written or detector-negative.
I would retain the reason it could not be analysed. A removed article and an image URL pose different follow-up tasks, even if both are absent from the classifier output.
The arithmetic follows the same principle as the AI visibility denominator guide. Here, the specific issue is coverage of source text.
A unique URL is not a measure of exposure
A source cited once and a source cited repeatedly can each contribute one unique URL.
So a share of unique sources does not tell me the share of answer passages, users or viewing time associated with those sources. Those questions require their own counts.
I would want repeated citation events available separately before making an exposure claim.
The paper did evaluate the detectors
It reports checks on 200 human texts and 200 generated texts. It also examines 105 articles from reported AI-content domains, while acknowledging that their individual authorship could not be verified. Those tests inform validation; they do not settle authorship across the audit sample.
I would not describe the work as having no detector evaluation. I also would not treat a successful test set as proof of perfect performance on every kind of web page.
A clean test does not remove deployment uncertainty
An observed zero-error result belongs to the examples tested.
Before applying it more broadly, I would ask whether the evaluation includes the kinds of text encountered in the audit. Edited drafts, short passages and mixed production workflows may call for different checks from a set of fully generated documents.
Those are validation questions I would raise, not findings that the detector failed on each of those categories.
A lower-bound claim needs assumptions
The authors interpret their estimate as a lower bound. I would retain that as their interpretation rather than a guaranteed minimum for true AI authorship.
Missed generated texts could make a detector count too low. False positives could move it in the other direction. Unanalysed pages add uncertainty whose direction cannot be resolved from their absence.
To turn a flagged share into a bound on actual authorship, I would want the assumptions and supporting error estimates stated for the relevant population.
Presence does not establish preference for AI-generated sources in AI search
Finding flagged pages among citations does not tell us that the engine preferred those pages because they were AI-generated.
For that question, we would need a comparison with the sources available to be selected, along with relevant differences between them. Topic fit, availability and other characteristics would need attention.
An audit of selected sources can show what appeared in that collection. It cannot, on its own, identify the causal role of the writing process.
That limits the publishing advice I would draw from it. I would not tell a business to generate more content on the strength of this result.
Authorship does not settle source quality
For a practitioner, the next useful check is the statement being supported.
Does the cited page substantiate the answer? Are its figures traceable? Has the answer preserved the conditions attached to them?
A detector flag can prompt additional review. Passing a detector should not exempt a page from that review.
This is my proposed source-review procedure, not an intervention tested by the audit.
A human-written source can repeat a shared assertion; an AI-assisted source can report original evidence. The two classifications should not be merged.
How I would describe this finding about AI-generated sources in AI search
I would preserve the classifier name and categories, the analysed-text count and the collection boundaries.
I would also separate any subsequent factual review from the authorship assessment. If a claim is wrong, the report should show the mismatch. If production remains uncertain, it should say so.
That lets a reader act on a specific concern without needing to accept an unsupported accusation about a publisher.
For a current audit, I would record the detector version and settings alongside the captured text. A later test on a changed page or a changed classifier is another observation, not a replacement for the earlier record.
The same applies to the search product. These findings do not establish today’s platform rates. The arXiv version date identifies the document reviewed; it should not be substituted for a verified collection window.
Frequently asked questions about AI-generated sources in AI search
Did the audit prove that 16% of all AI search citations were AI-written?
No. The reported share concerns likely-AI detector labels among successfully scraped unique cited URLs in the analysed sample.
Does an AI detector flag prove a publisher used AI?
It is classification evidence. A claim about the actual production process needs further support.
Does the study establish that AI-written pages are inaccurate?
No. Authorship classification and factual accuracy are separate assessments.
Does appearing in the cited-source sample mean AI content gets a ranking advantage?
No. Presence among selected sources does not establish a preference or the effect of changing how content is written.
Should a detector-negative page be trusted automatically?
No. Its claims still need evidence. A negative label is not a factual verification.
The GEO Lab takeaway
Keep the detector result attached to the text it assessed, and keep the missing sources visible in the report.
Then check the claim the citation is supposed to support. That gives a reader a specific finding to inspect, rather than an authorship label doing the work of a source review.
Source and evidence locators
Allaham, M. and Diakopoulos, N. (2026), Synthetic Sources?: Auditing Generative Search Engine Citations for Evidence of AI-Generated Sources. arXiv:2605.23684v1.
- Source collection and scraping: Methodology.
- Classified subset and category counts: Results, “Citation of AI-generated Sources”, Figure 1.
- Detector evaluation: Methodology and Appendix A.1.
- Coverage and validation boundaries: Limitations.
- arXiv record and publication information
Have questions about this topic? Contact The GEO Lab · Return to homepage