Geolify Build Log: Scoring What We Can’t Fully See

Field notes on building the GEO Score: a 0–100 composite over six pillars that has to stay honest about observing systems from the outside.

The Honesty Problem Inside a Single Number

AI answer engines are black boxes. ChatGPT, Perplexity, Gemini, and Claude do not publish ranking functions, citation criteria, or crawl schedules. Geolify audits how these engines talk about a site — what they can reach, what they can parse, what surfaces exist for them to quote — and every observation arrives through the output side of a system whose internals we cannot inspect.

A composite score over that kind of system carries a built-in temptation: to sound more precise than the observation underneath supports. A 0–100 number looks like a measurement, and measurements get read as ranks — as statements about where a site sits inside an engine. That reading is not available to us. Building the GEO Score meant deciding, repeatedly and in writing, what the number may say and what it must refuse to say.

What the Score Actually Measures

The score is a weighted composite over six pillars: AI Indexability, AI Crawler Governance, Performance and Technical Health, Structured Data, AI Readiness, and Static Content. Each pillar scores evidence we can observe directly — whether declared crawler policy permits AI access, whether structured data is present and parseable, whether the content an engine would quote exists in static, fetchable form.

observable evidence → six pillar scores → weighted composite (0–100) → prioritized fixes

The decomposition matters more than the weights. A score that cannot be taken apart is an opinion; a score that decomposes into checkable evidence is a diagnosis. Every point the composite awards or withholds traces back to a specific artifact a team can inspect — a file, a header, a schema block. That traceability is what makes a re-audit meaningful: run the same measurement again after a fix and the delta is evidence, not vibe.

The Action Center exists for the same reason. An audit produces findings; a team needs an ordered list. Each Action Center item carries an issue, a recommendation, and an expected impact, so the path from score to work is mechanical rather than interpretive.

What Did Not Work

The first version reported the composite beside a flat findings list. Operators saw a number and a wall of issues with no ordering. The score summarized but could not prioritize, and prioritization is most of what a score is for. The fix was the Action Center structure — findings became a queue, and the queue became the unit of work.

The second failure was more interesting. We tried grounding the score in engine answers — tracking whether responses cited the site and folding that back into the number. Two problems surfaced fast. Answers are unstable across identical prompts, so the same site could score differently on the same day with nothing changed. And presence-in-answer conflates surface quality with engine behavior we do not control — a model refresh could move the number without the site doing anything.

The resolution was a separation of instruments. The score measures the surface: what the site declares, serves, and structures. Prompt Monitor tracks prompts and brand mentions between audits: what engines actually said. Each instrument stays honest about what it observes, and neither contaminates the other.

Weighting was the third correction. Equal weights treated crawler governance as one-sixth of the story; in practice, access failures gate everything downstream. An engine that cannot fetch a page cannot cite it, whatever the content quality underneath. The weights now reflect gating order rather than symmetry.

What the Score Does Not Claim

• It is not a rank. No external tool can observe a site’s position inside an engine’s synthesis the way a search results position was once observable. The score does not claim one.
• It is not a promise of citation. A high score means the evidence surface is in order. Whether an engine cites the site remains the engine’s decision.
• It is not a universal leaderboard. Comparisons are meaningful for the same site across audits, and directionally between similar sites — not as an absolute ordering across the web.
• It is not stable ground truth. Engines change behavior without notice. A score describes the surface at audit time; the weekly re-audit exists because both the surface and the engines drift.

Why the Boundary Is the Feature

A bigger claim would be easy to write and wrong to ship. The reason is operational, not only ethical: a score that overclaims teaches teams to chase the wrong work. If the number implied engine internals, teams would optimize for the number instead of the surface, and the audit would stop being a measurement and start being a target.

Bounded claims keep the loop honest. The score’s job is narrow: make a weekly decision cheap. What is broken, what to fix first, and whether the last fix moved anything.