If someone sells you AI visibility tracking, ask one thing first: how many times did they run each question?
Start here
AI tools don't answer the same question the same way twice. That changes how you have to measure.
What the research says
A 2026 study by Rand Fishkin of SparkToro and Patrick O'Donnell of Gumshoe.ai had 600 volunteers run 12 prompts through ChatGPT, Claude and Google's AI, nearly 3,000 runs in total. The odds of getting the same list of brands twice were under 1 in 100. The odds of the same list in the same order were closer to 1 in 1,000.
So tracking your "position" in an AI answer doesn't tell you much. Your position on Tuesday won't be your position on Wednesday.
But brand visibility held up better. For some brands, in some categories, the same names appeared in 60% to 90% of responses for a given kind of question, even as their order changed. Smaller markets were steadier than big ones.
The article's conclusion: track how often a brand appears across many prompts and many runs. It called that imperfect, but closer to reality than treating AI answers like rankings.
What's still unknown
The study left open how many runs you need for a reliable number, whether API results match what real users see, and how many prompts it takes to represent a market. I'm not going to invent a standard for those. Anyone who gives you one as settled is guessing too.
A method you can run yourself
- Build your question list. Twenty to thirty questions your buyers actually ask, per service you want more work for. Use sales calls, emails and the kind of queries people type. That number is my judgment as a starting point, not a standard.
- Run each one several times, in each tool you care about, on a set date. Use a clean session.
- Record the answer. For each run: were you named, were you described correctly, were you linked, and which sources did it cite?
- Report it as a share of runs. "Named in 14 of 30 runs" is a number you can compare next month. Always log the question, the tool, the region and the date.
- Repeat on a schedule. Monthly is enough to see a direction.
Tools can automate the repetition. If you use one, check that it shows you the raw answers, how many runs sit behind each number, and the dates. A single score with none of that is hard to trust.
Connect it to the business
Visibility isn't the goal. Enquiries are.
- Ask every new contact how they found you. Add a field for it. Include "an AI tool" as an option.
- Look at referral traffic in your analytics for visits that arrive from AI tools.
- Keep measuring classic search. Search Console shows clicks, impressions, click-through rate and average position for Google results.
AI-driven visits will be a small number at first. Watch the direction, not the size.
What I'd ignore
A single answer you got once. A dashboard with no dates. Anything that treats position in an AI answer as a stable ranking.
Sources
- Search Engine Land, AI recommendation lists rarely repeat, SparkToro study, January 2026.
- Google Search Console Help, Performance report (Search results).
Common questions
Is ChatGPT prompt tracking reliable?
A single run is not. A 2026 SparkToro study found the odds of the same list of brands twice were under 1 in 100. A share of appearances across many runs is closer to reality.
How many prompts should an AI visibility audit track?
There is no published standard. Twenty to thirty questions per service is a starting point, as a judgment, not a rule.
How do I measure leads from AI search?
Ask each new contact how they found you, check referral traffic in your analytics for visits from AI tools, and keep tracking classic search in Search Console.
Kryston's AI visibility audit gives you a dated baseline of this for your own firm: what ChatGPT, Perplexity, Gemini, AI Overviews and Google say about it today, and what to fix first.
Kryston guides
Analysis for IT services and software development firms on search, AI visibility and buyer trust.
