Measuring Answer Share Between Audits

Answer share — how often tracked prompts return a mention or citation — is measurable between audits only as a trend. A fixed prompt set and repeated runs separate real movement from generative noise.

Defining the Metric

Answer share is the proportion of a defined prompt set whose responses mention or cite the entity. It is prompt-level by construction: answer engines do not rank pages, they produce answers, so visibility exists or fails one answer at a time. The metric counts appearances across a fixed set of questions where the entity has a legitimate claim to appear — category questions, comparison questions, branded questions.

Two things must be recorded per response, not one: whether the entity appeared at all, and how. Mentioned in passing, cited as a source, or cited and described correctly are different outcomes with different values. The second layer is where the information lives; an incorrect mention is not visibility, and an uncited mention is not attribution.

The prompt set is the instrument, and instrument design decides what the metric can see. Category questions test whether the entity appears where it should compete; branded questions test whether it is described correctly where it is already known; comparison questions test both. A set skewed toward branded prompts manufactures flattering share; a set weighted toward category prompts measures the harder contest.

Why a Snapshot Misleads

Generative answers are non-deterministic. The same prompt, run twice, can return different citation sets — the retrieval layer samples, the synthesis varies, the supporting sources shift. A single run is a draw from a distribution, not a measurement of it, and treating one draw as a score manufactures confidence from variance.

The baseline moves as well. Model versions change, retrieval corpora churn, new pages enter the index and old ones decay. A snapshot taken this week and a snapshot taken next week are draws from two different distributions; comparing them attributes the world's movement to the site's.

The Fixed Prompt-Set Method

The method that turns answer share into measurement has three requirements.

• Fix the prompt set. Select the questions where the entity should appear — enough to cover the categories that matter, few enough to run repeatedly. The set stays constant; changing the questions changes the instrument.
• Repeat the runs. Each prompt is run multiple times across the engines being tracked — for a Geolify audit, ChatGPT, Perplexity, Gemini, and Claude — because a single run cannot distinguish a stable absence from an unlucky draw.
• Record at the claim level. Per prompt and per run: mentioned or not, cited or not, described correctly or not. The record has to stay granular enough to tell "appeared and wrong" apart from "appeared and right."

Between full audit runs, this prompt-level tracking supplies the continuous signal. It is the function Prompt Monitor serves inside Geolify's weekly loop — watching brand mentions and citations in the interval the audits bracket, so a surface fix shipped on Monday can be watched in the answers through the week.

Separating Movement from Noise

One prompt flipping in one run is noise. Movement earns the name by surviving three tests.

• Persistence: the change holds across repeated runs rather than appearing once.
• Breadth: it appears across engines or across related prompts, not in a single cell of the grid.
• Explicability: it follows a change that was actually shipped — a fixed surface, a corrected description, a new citable page.

A change that passes all three tests is signal worth acting on; a change that fails any of them is variance worth ignoring. The discipline cuts both ways: a favorable jump that fails the tests is as meaningless as an unfavorable one, and acting on either wastes the loop.

The Trend Line Is the Metric

What counts is the trend's direction over weeks, laid against the record of what shipped. An audit establishes where the six surfaces stand; prompt-level tracking shows whether the fixes moved the answers. That pairing is the weekly loop made legible: measure, fix, measure again.

A trend line also disciplines reporting. Placed beside a ship log, every movement forces the question "what did we change" — and movements with no answer get treated as noise until they persist. The record keeps visibility work honest about cause.

The honest limits belong in the method. Correlation between shipped fixes and rising answer share is directional evidence, not proof of causation on any single prompt — too many variables move at once. What the trend line offers is better than a snapshot and humbler than a guarantee: a falsifiable record of whether the answers are moving in the direction the work predicts.