You Can Measure AI Visibility. You Can't Measure Much Else Yet.

Last Thursday's newsletter carried a Signal about ChatGPT traffic hiding in "direct". Since GPT-6 launched, links in ChatGPT answers had stopped carrying the utm_source=chatgpt.com tag, and analytics tools were filing those visits as direct traffic. It was reported widely, OpenAI hadn't commented, and it looked like a real change.

Within a day it turned out to be a bug. Cedric Huesler at Adobe, whose analysis had first flagged the drop, updated his post on 15 October to say the tags were coming back. Webflow's own data showed an average 47% fall in ChatGPT-attributed traffic across its customer sites between 8 and 15 October, with tagging restored on the 15th.

Nobody lost visitors. Plenty of people will have lost a week of reporting, and some will have spent time explaining a drop that never happened. That's a useful illustration of where AI search measurement actually is right now.

In last week's post I listed five questions to ask before you buy AEO. The third was: how will you measure it, and what can't you measure? That's the one people find hardest to answer honestly, so this post answers it properly.

What you can measure directly

There is more first-party data than there was a year ago. It all measures visibility. None of it measures outcomes.

Google Search Console. The Generative AI performance report has been available to all sites worldwide since 31 August. It shows impressions in AI Overviews and AI Mode, broken down by page, country, device and date, with a separate report for Discover. It doesn't show clicks, and it doesn't show the queries that triggered an appearance. It tells you that you were seen, not what happened next.

Bing Webmaster Tools. Microsoft's AI Performance report, in public preview since February, counts citations across Copilot, Bing's AI summaries and unnamed partner integrations, along with the cited pages and the "grounding queries" Bing used to find them. It's useful, and in some ways more detailed than Google's. It also covers one ecosystem only, and has no click or traffic data.

Analytics. GA4 can show visits referred by ChatGPT, Perplexity, Gemini and others, as long as the platform passes a referrer or a tag. As October showed, that depends entirely on the platform choosing to, and continuing to, label its links. AI referral numbers are a floor, not a total. I wrote about the wider version of this problem in Your Traffic Numbers Are Not Wrong. They're Just Incomplete.

What you can only estimate

Most AI visibility tools work the same way. They take a list of prompts, run them through ChatGPT, Gemini, Perplexity and AI Mode on a schedule, and record which brands and pages show up. The output is a visibility or share-of-voice score.

That's sampling, not observation. The tool sees the answers it asked for, not the answers real people got. AI responses vary from one run to the next, by location, by account and by how the question is phrased. A score built on twenty prompts run once a week can swing for reasons that have nothing to do with your site.

None of that makes these tools useless. Used well, they show direction of travel and how you compare with competitors on the questions that matter to you. Used well means:

• A fixed set of prompts that reflect how your customers actually ask, agreed up front and changed rarely
• Repeated runs, so one odd answer doesn't become a headline
• Trends over months rather than week-to-week movement
• The same tool and settings throughout, because scores from different tools aren't comparable

The tools I'd consider are in Your 2026 reporting stack. What's changed since June is less the tools than how much weight their numbers should carry.

What you can't measure yet

This is the part that gets left out of sales decks.

Clicks from Google's AI features. Search Console's AI report counts impressions only, and Google hasn't given a timeline for click data. Those clicks still sit inside your overall Search numbers. You just can't separate them.

Influence without a click. Someone reads an AI answer that mentions you, closes it, and searches your name a week later. Or rings you. Or mentions you to a colleague. No tool connects those dots, and this is probably where most AI search value sits for B2B and local businesses.

What happens inside the platforms. Google's own guide to generative AI search is direct about this: no third-party tool has access to its internal ranking or AI systems. The same applies to OpenAI and everyone else. Anyone claiming to know exactly why you were or weren't cited is guessing, however good the dashboard looks.

What an honest AI visibility report looks like

The fix isn't a better tool. It's labelling every number for what it is, so nobody mistakes a sample for a fact.

Observed: Search Console AI impressions, Bing AI citations and GA4 AI referrals. These tell you that you were seen or visited, on the platforms that report it.

Estimated: prompt-sampled visibility and share-of-voice scores. These show direction of travel and your position against competitors.

Directional: branded search trends, "how did you hear about us?" answers on enquiry forms, and enquiry quality. These suggest whether visibility is turning into interest.

Two more things belong in every report. First, a check on accuracy: what do the main AI platforms actually say about the business, and is it right? A citation that gets your services or prices wrong isn't a win. Second, a log of known data problems, like the 8 to 15 October ChatGPT gap. Annotate it once and nobody has to explain it again in three months.

On cadence, monthly is plenty. Weekly AI visibility numbers mostly measure noise.

For agencies: selling it without overpromising

If you sell AI visibility, the measurement gap is a pricing problem as much as a reporting one. You can't sell a guaranteed citation, because neither Google nor OpenAI guarantees inclusion. You can't price on AI-driven revenue, because most of it can't be traced.

What you can do is price the work: the technical fixes, the content, the business information cleanup, the reporting. Then report honestly on what moved, what didn't, and what's still unknown. Clients tend to trust that more than a score that only ever goes up.

This is exactly the question I'm taking to DudaCon '26 on 28 October, where I'm hosting the Day 3 sessions. The first is The Value of Visibility: How to Sell Local AEO, with Naama Mizrahi from Duda and Ruan Marinho of Develomark and SplashDash. How do agencies measure AI visibility fairly for a local business, and price it without promising what nobody can deliver? It's free, runs 10am to 1pm PT (5pm to 8pm UK), and registering gets you the recordings.

Measure what you can, label the rest

AI search measurement is better than it was six months ago, and it will be better again in six months. Right now you can see whether you're being shown and cited on Google and Bing. You can estimate how you compare elsewhere. You can't yet prove what that visibility is worth, and anyone who says otherwise is selling something.

That's not a reason to ignore it. It's a reason to report it honestly, keep the fundamentals in order, and be wary of any number that looks too tidy.

If you'd like a clearer picture of where your own site stands, that's where my digital consultancy work starts.