Why do AI visibility measurements vary?

AI answers can vary across prompts and systems, so repeated testing gives a more useful sample than one personal chatbot search.

For B2B marketers, this is one part of choosing a useful video plan. Start with your buyers’ questions before committing production time.

Quick Answer

• Dane asks how variable answers can support measurement.
• Adrian recommends repeated tests across models and prompts.

This fits inside a bigger picture. If you want the full view first, here's the bigger picture.

What this means for your video plan

Variation is part of the measurement problem Dane raises. If two answers differ, how can a marketing team use the data to make a decision? Adrian’s answer is to test more than one prompt, across different systems, repeatedly.

That does not turn AI answers into a fixed ranking. It creates a larger set of observations from which the team can examine patterns. The question becomes whether a brand appears consistently within the tracked conditions, rather than whether it appeared once on one person’s computer.

The full interview provides the context behind the number discussed in the clip. The 28% figure belongs to one selected prompt and the tracked answers in the dashboard’s review window. It is not a market-wide visibility score or a rate of buyer conversion.

Averages can help, but their meaning depends on what was sampled. The prompts, systems, dates, and tracking conditions remain part of the result. Changing those conditions can change what a comparison means.

For a B2B content team, this is a reason to document the baseline before publishing. Then keep watching the same questions while recording changes to the content and distribution. That is a suggested experimental practice, not a claim that this interview demonstrates causation.

Dane wants evidence strong enough to guide investment. The discussion supports careful observation. It does not support promising a specific citation gain or new deal from a visibility percentage alone.

The result should also stay attached to the question it measures. A percentage for a narrowly defined service and location cannot stand in for the whole business. The full audit includes different prompts with different outcomes. That is why the next planning decision needs the question-level detail as well as the overall picture. An apparent improvement is a reason to inspect the evidence, not declare a campaign successful.

Doing the thing is the easy part. The hard part is knowing where you're already winning and where you're invisible. That's exactly what our free AI Visibility Snapshot shows you, before you spend a dollar.

Frequently Asked Questions

Is one personal ChatGPT search a reliable market baseline?

It is a single observation under personal conditions. Adrian recommends a broader, repeated prompt set.

What does the 28 percent figure in the interview measure?

It describes brand mentions for one selected prompt across the audit’s tracked answers and review window. It does not measure revenue or all AI searches.

Do repeated tests prove my content caused an improvement?

No. They help describe patterns. Establishing causation or commercial value requires additional evidence.

If you need help choosing the buyer questions your video should answer, talk with Dane.

Clip Transcript Source

Riverside source excerpt corresponding to this clip. The edit may differ slightly.

So for the naysayers about AI visibility measurement tools, like you know, there's n I I've heard some you know pushback and lack of consensus at times about like how scientific this is, because you can't really measure AI visibility because you get different answers. It's it's it's what probabilistic, not deterministic, right? So what what we're saying here is because of that, the methodology of measuring across these different platforms multiple times, like you know, every single day, that

Adrian: Yep.

Dane: that body of knowledge, bot body of data gives us some sort of like reasonable, defensible measurement system rather than just someone measuring on chat GPT on their personal computer. Is that is that the takeaway here? Okay.

Adrian: Exactly. Yep, exactly. And by testing different models, different prompts, running them every day, getting a large quantity of data, we can have some more faith that the kind of the average of that is 28 % across all of those is close to the reality that's being experienced out there in the world.

Most teams don't get this right on the first try. Here's how we approached it for a real team. Curious what it'd look like for you? Let's talk it through.