Would AI give you the same answer if you asked again?
Ask AI the same question twice and you can get two different answers. We ask each question many times, so one lucky answer doesn't fool you.
What you do with it
- Report the questions where AI's answer about you is steady
- Read every answer where AI contradicts itself about you
- Run 20 samples on the questions closest to revenue
What it can't do
- A steady answer isn't always a correct one. AI can repeat the same wrong fact every time.
- More samples give a surer answer, but use more of your plan. You choose 5, 10 or 20.
- We measure one assistant at a time. ChatGPT can be steady while Gemini isn't.
- A steady answer today can still change when the assistant or its sources change.
Why AI answers change from run to run
An AI assistant doesn't look up a stored answer. It writes a new one each time. Two runs of the same question can name different companies, quote different pages and place you differently.
One run tells you what happened once. That cuts both ways. One bad run can send a team rewriting pages that were fine. One good run can make a problem look solved when it isn't.
How AI answer consistency is measured
Pick a depth: 5, 10 or 20 samples. Set it once for your brand and change it for the questions that matter most. We ask the same question to the same assistant that many times. Every answer is kept in full.
Each check reports a consistency percentage and a variance score out of ten. It shows your average, lowest and highest visibility across the samples. It shows how the answers split between positive, neutral and negative. And it counts how many samples had a false claim about you.
An average built from 38% and 42% is a different story from one built from 5% and 75%. That's why you see the spread, not just the average.
The four levels, and the one to act on
Consistent: AI says roughly the same thing every time. You can plan around it.
Minor: small changes in wording and order, same substance. Nothing to chase.
Moderate: answers differ enough to change what a buyer takes away. Worth reading the samples.
Contradictory: AI gave conflicting answers about you. This usually means the pages it reads disagree. Fixing those pages matters more than rewriting your own.
Frequently Asked Questions
Questions about answer consistency
AI assistants write a new answer each time instead of looking one up. Two runs can name different companies and quote different pages. That's normal, and it's why one answer can mislead you.
It's a percentage showing how closely the samples agreed. High means your position is steady and you can plan around it. Low means AI has no settled view of you yet.
Five catches obvious swings. Ten is a good default. Twenty is worth it for questions tied to revenue. Set a default for your brand and change it per question.
A wrong fact in 1 of 20 answers is a different problem from one in 18 of 20. Sampling tells you which you have, so you know how urgent the fix is.
Starter and above ask each question many times. On the Basic plan, it covers ChatGPT and Gemini.
Find out how steady your AI visibility is
Start with a free AI visibility audit, then check the questions that matter most.
Free AI visibility audit
Find out what ChatGPT tells your buyers about you.
Where you're named, what AI gets wrong, and the first three things to fix. Emailed within two hours.
- Within 2 hours
- No credit card needed
- Free