What a Zero-Citation Experiment Can and Cannot Tell Us

By Artur Ferreira · The GEO Lab

Methods review · EDX, E089 and E016 · Limits of inference

My entity-density test ended with zero citations on all 22 competition queries. The report went further: it described domain authority as the gate that had prevented page-level signals from operating.

Reading the result again, I would separate those two statements more carefully.

The zero is in the recorded output. The gate is an explanation for it. The collection did not directly observe that gate, and it did not vary domain authority to test whether changing it would change the outcome.

That leaves a useful result, with a narrower conclusion than some of my original wording suggested.

This post is about how I’d read that kind of evidence now, including when a zero-citation experiment leaves its original question unresolved.

What the EDX zero-citation experiment actually observed

The EDX experiment report describes ten pages on thegeolab.net, spanning 12.9–29.5 unique named entities per 1,000 words. On 7 April 2026, the competition phase submitted 22 queries to Perplexity sonar-pro through DataForSEO, with web search enabled.

Each query had a predefined set of pages judged plausible answers. None of the queries cited a target page. One query, CQ19, was interpreted as being about academic citation and was excluded from substantive interpretation. The other 21 also returned no target citations.

The deposited EDX preprint, sections 3.4–3.6 and 4.2, gives the protocol and the per-page results.

This was a comparison of existing pages with different measured densities. It did not randomly assign density changes to otherwise identical pages. Even a positive association would therefore have needed care before becoming advice to add more named entities.

Why an undefined correlation is different from a zero correlation in a zero-citation experiment

The planned analysis asked whether pages with higher entity density also had higher competition citation rates.

Every page’s observed rate was zero. There was variation in density, but none in the outcome being correlated with it. Pearson and Spearman correlations were therefore undefined.

Writing the coefficient as zero would change the meaning. It would suggest a calculated absence of association. Here, the intended statistic could not be calculated meaningfully from those outcomes.

The observed citation rate can still be reported. The limitation applies to the planned correlation, not to every possible analysis of an all-zero dataset.

Different pages also appeared in different numbers of plausible-answer sets. The report records between five and fourteen opportunities per page. Those overlapping sets are not independent experiments: several pages share the same answer capture.

Ten rows of zeros do not turn one query collection into ten independent replications.

A zero-citation experiment result doesn’t locate the failure

The EDX report interprets the result through an authority-gate hypothesis. That remains a possible explanation. Citation absence alone cannot distinguish it from every competing explanation.

For example, the target pages might not have entered the system’s candidate set. They might have been available but not selected for attribution. The query might have been interpreted differently from the researcher’s intended topic.

Those possibilities call for different evidence. A visible citation list tells us which sources the answer attributed. It does not expose every document considered or every selection decision.

The original report’s claim that the result would replicate on other platforms was also too strong. Only the Perplexity run was completed. Other platforms would need their own observations.

I would keep EDX’s recorded null outcome while qualifying its explanation: the collection found no target citations and could not estimate the planned density correlation. It motivated the authority-gate hypothesis; it did not isolate or verify that mechanism. The Extractability layer in the GEO Stack addresses passage-level selection, but the question of whether a page enters the candidate set at all sits upstream of extraction.

For the separate question of what a citation checker can detect, see AI citation tracking and measurement coverage. This article assumes the captured absence has been checked and asks what follows from it.

Zero observed citations still contain information

An all-zero result from a zero-citation experiment does not make the work worthless. It records that the expected event did not appear in a defined collection. That can challenge a claim that a technique should reliably produce citations in those conditions.

It also helps plan the next test. If the intended analysis needs differences between pages, an all-zero pilot warns that the current setup may not supply them.

The claim needs to stay attached to the collection. “No citations appeared in these checks” leaves room for a different query set, product or date to produce another outcome. “This page cannot be cited” extends beyond the observations.

A finite run of zeros also does not establish that the underlying chance of citation is exactly zero. How tightly it constrains that chance depends on the sampling assumptions and on how much independent information the checks provide. The GEO Stack framework treats citation as a downstream outcome of multiple layers; a zero result at the citation stage does not identify which layer failed.

Repeating the same query many times may answer a stability question. It does not automatically provide the same information as testing many different questions. The unit of inference needs to be chosen before treating the total number of calls as the sample size.

E089 shows a different way a zero-citation experiment comparison can become inconclusive

The E089 front-loaded-statistics report began with twelve posts arranged as six A/B pairs. Five pairs failed validity checks, leaving P3 as the only retained pair.

In that pair, none of four answer surfaces reconstructed the engineered opening’s statistic. They reproduced the plain declarative definition instead. The report identifies one timepoint, no D14 baseline for P3 and a sample below the Lab’s minimum-N standard.

That is an observation about reconstruction in a surviving pair. It is not an all-zero citation result across the original cohort. It also does not establish that front-loaded statistics generally help or harm citation rates.

The missing comparisons cannot be filled with zeros. An invalid pair failed to answer the intended question; it did not demonstrate that the treatment lost.

I’d preserve the original six-pair plan alongside the account of what survived. Otherwise, a reader sees a result from one pair without seeing how the planned comparison became so small.

A decision threshold cannot repair an uninformative zero-citation experiment comparison

E016 introduced a 22-percentage-point operational threshold that later Lab protocols adopted. The deposited E016 package explains its construction in paper.md, section 4.5: a reported 20% maximum combined citation rate plus a two-point buffer.

I would describe that as the study’s decision rule, not a universally calibrated minimum detectable effect.

A rate level and uncertainty in a change are different quantities. Establishing how much a before-and-after difference can vary requires evidence about that difference under the intended design, including repeated queries and the way observations are paired or clustered.

The package also discloses that Google AI Overview values for Days 3–5 were assumed rather than manually collected, in section 3.1. Those entries should not be presented as additional observed absences.

Neither a threshold nor a larger-looking denominator restores missing comparisons. If the intended correlation is undefined, a threshold does not supply it. If five pairs are invalid, it does not make them valid.

The threshold can remain part of a historical protocol record while its statistical interpretation is reviewed.

What I’d put in a zero-citation experiment report

I’d want the report to make the observed absence easy to verify and the unresolved question easy to find.

For EDX, I would write:

No target citation appeared in the 22 competition-query captures. One query was topic-invalid; the other 21 also produced no target citations. All target-page competition rates were zero, leaving the planned density correlations undefined. The mechanism behind the absence remains unresolved.

That wording retains the result without asking it to explain more than the collection measured.

Plan for an all-zero outcome before collecting

A pilot can check whether the capture method detects known citations and whether the proposed questions address the intended topic. If a detector handles redirects or source wrappers, the positive control needs to exercise that same path.

Choose any eligibility rule before the main analysis. Selecting only pages that happened to earn citations afterwards would change the question and the population being described.

If zeros remain, retain them. Report the planned analysis that could not be completed, then design a follow-up around the unresolved question. A follow-up that starts with regularly cited pages can study selection among those pages, but cannot establish how an uncited page becomes cited. For the broader workflow of planning such measurements, the query-set reporting guide covers that sampling question separately.

Frequently asked questions