What I learned building an AI visibility checker for a PR agency

I built the thing that runs our free AI visibility check. While testing it, I ran 20 questions through four AI tools. These are the things that surprised me, the things that annoyed me, and what I’d tell any brand that asked.

I’m the co-founder at JBH who builds things rather than pitches things. Aran will tell you I’m happiest with a terminal open and a half-finished prototype. So when it became obvious that our clients needed to know what ChatGPT and Google AI were saying about them, I did what I always do: I built a tool rather than reading a report about one.

That tool now runs our free AI visibility check. This post is about what fell out of it once it was running, because some of that was surprising and most of it was useful.

Why I built it instead of buying one

There are plenty of GEO dashboards out there now. Some are decent. But I wanted to see how the answers were being built, rather than a score. So the brief I gave myself was dead simple. Take the real questions a brand’s customers ask, the ones they type before they know the brand exists. Run the same 20 through ChatGPT, Gemini, Perplexity and Google’s AI surfaces. Record who gets named, how each brand is described, and which sources the answer cites. Then do it again next month and show the difference.

It isn’t a ranking tool. There’s no position one. There’s a share of answers, a set of descriptions and a list of sources. That turned out to be the right shape, because what matters is whether you’re in the answer and what it says.

None of them agree, and that’s the point

First surprise: ask four AI tools the same buying question and you often get four different shortlists. Not wildly different, there’s usually a brand or two that everyone names, but the rest of the list moves. Different training data, different live search, different ideas about what counts as a trustworthy source.

At first I took this as a flaw in the tools. Now I think it shows the gap. Semrush looked at more than a thousand ChatGPT product categories in the first half of 2026 and found only about one in seven had a clear brand owner. The rest are still open to any competitor that can build consistent visibility. When the AIs disagree, nobody has won yet, and that’s an opportunity.

It also means you can’t check one tool and call it done. Previsible’s 2026 report found Google’s AI Overviews and AI Mode together drive more AI-influenced traffic than every standalone assistant combined, while ChatGPT sends the vast majority of the standalone referrals you can track. Two very different places, two very different answer-builders. You need both.

The wording of the question changes the answer

Second surprise, and this one annoyed me. “Best dog food subscription UK” and “best dog food subscription for a fussy eater” and “is Tails.com any good” produce three different answers with three different casts. Obvious in hindsight. But it means a single screenshot of ChatGPT naming (or ignoring) your brand is almost meaningless, and I’ve seen a lot of strategy decisions made on exactly that.

What works is a fixed set of real customer questions, phrased the way customers phrase them, run the same way every month. Then you’re measuring a trend in your share of answers, not reacting to one lucky or unlucky prompt. The first run is a baseline. The fourth run is where it gets interesting.

The sources are weirder than you think

This was the most useful thing the tool showed me, and the thing I’d want every PR person to see. The sources behind the answers are not the sources a PR plan would target.

Yes, there’s journalism. Muck Rack’s May 2026 study of millions of AI citations found about a quarter are journalism and more than four in five are earned media overall. But alongside the nationals you get a Reddit thread from a specific subreddit, a Wikipedia page, a specialist blog that ranks for nothing, a competitor’s own comparison page, and a round-up that was written years ago and never updated. Ahrefs found that fewer than four in 10 of the pages Google’s AI Overviews cite appear in the top 10 results, so most of what gets quoted isn’t even on page one.

And the journalists? Muck Rack’s December 2025 report compared the journalists brands pitch most with the journalists AI engines actually cite for those brands. The overlap was a couple of per cent. Which lines up with what I saw: the coverage a PR dashboard counts (domain authority, link, done) and the coverage a model reads are different lists with a bit of overlap. The dashboard is fine; it was built to measure something else.

Stop optimising for the robot

Once people see this stuff the temptation is to start gaming it. Write “GEO content”. Stuff 40 FAQs onto a page. Hire someone who calls themselves a prompt whisperer. I’ve watched it happen and I’m not going to pretend I wasn’t tempted myself, because building tools makes you think everything is a tools problem.

The reality is the model quotes what’s quotable: a clear sentence, from a source it trusts, that answers the question. The research keeps pointing the same way. Ahrefs found web mentions track AI Overview visibility about three times more closely than backlinks. Seer Interactive found page-one Google rankings were the strongest predictor of being mentioned in ChatGPT answers, while backlinks on their own were weak or neutral. And when Semrush looked at what separated the brands that owned a ChatGPT category from those that didn’t, the only statistically significant difference was branded search volume. People searching for you by name.

There’s no hack in any of that. You need people talking about you, rankings for what you sell and a name people search for. If you’re worth quoting and you get quoted in the right places, the AI tends to follow.

What I’d tell any brand

Run a baseline. 20 real questions, four tools, written down. Don’t guess, because I guarantee the answers will surprise you, and not always in a bad way. Then check the boring plumbing: are you blocking AI crawlers, does your site say plainly what you are, are the facts about you consistent everywhere. Then take the list of sources the tools showed you and point your PR at those, specifically, rather than at the biggest domain authority you can get. Re-run it monthly and watch the share of answers and the descriptions move.

And keep some humility about it. I built the thing and the answers still surprise me every month. So be wary of anyone who promises certainty here.

If you’d like to see your own baseline, the free AI visibility check asks ChatGPT 10 of your customers’ questions, with a strategist’s write-up: what we found and what we’d fix first.

Questions people ask

Is the JBH AI visibility check automated?
The questions are run by our tool in ChatGPT, so the answers are real and current. A strategist then reads every answer and writes up where you stand and what to fix first. The tool does the legwork, a person does the thinking.
Which AI tools does it cover?
ChatGPT, Gemini, Perplexity and Google’s AI surfaces (AI Overviews and AI Mode). That covers the standalone assistant that sends most referral traffic and the Google surfaces where most AI-influenced discovery happens.
How often should we re-run it?
Monthly, with the same questions. A single snapshot tells you very little because the answers move. The trend in your share of answers over three or four months is the number that means something.

Win the budget from your finance director

A short deck written for finance, and a calculator that models the return with your own numbers. No prices, just the case.

Privacy policy

What does AI say about your brand today?

Our free check asks ChatGPT 10 real customer questions and shows you where you stand straight away.