We know AI responses are probabilistic \u2013 if you ask an AI the same question 10 times, you\u2019ll get 10 different responses.<\/p>
But how<\/strong> different are the responses?<\/p> That\u2019s the question Rand Fishkin explored in some interesting research.<\/p> And it has big implications for how we should think about tracking AI visibility for brands.<\/p> In his research, he tested prompts asking for recommendations in all sorts of products and services, including everything from chef\u2019s knives to cancer care hospitals and Volvo dealerships in Los Angeles.<\/p> Basically, he found that:<\/p> For my research, as always, I\u2019m focusing exclusively on B2B use cases. Plus, I\u2019m building on Fishkin\u2019s work by addressing these additional questions:<\/p> To explore those questions, I first designed 12 prompts:<\/p> I ran the 12 prompts 100 times, each, through the logged-out, free version of ChatGPT at chatgpt.com (i.e., not the API). I used a different IP address for each of the 1,200 interactions to simulate 1,200 different users starting new conversations.<\/p> Limitations<\/strong>: This research only covers responses from ChatGPT. But given the patterns in Fishkin\u2019s results and the similar probabilistic nature of LLMs, you can probably generalize the directional (not absolute value) findings below to most\/all AIs.<\/p> Your customers search everywhere. Make sure your brand shows up<\/span>. <\/p> The SEO toolkit you know, plus the AI visibility data you need. <\/p> <\/p><\/div> Start Free Trial<\/span> <\/p> Get started with<\/p> So what happens when 100 different people submit the same prompt to ChatGPT, asking for product recommendations?<\/p> On average, ChatGPT will mention 44 brands across 100 different responses. But one of the response sets included as many as 95 brands \u2013 it really depends on the category.<\/p> Competitive vs. niche categories<\/strong><\/p> On that note, for prompts covering competitive categories, ChatGPT mentions about twice as many brands per 100 responses compared to the responses to prompts covering \u201cniche\u201d categories. (This lines up with the criteria I used to select the categories I studied.)<\/p> Simple vs. nuanced prompts<\/strong><\/p> On average, ChatGPT mentioned slightly fewer brands in response to nuanced prompts. But this wasn\u2019t a consistent pattern \u2013 for any given software category, sometimes nuanced questions ended up with more brands mentioned, and sometimes simple questions did.<\/p> This was a bit surprising, since I expected more specific requests (e.g., \u201cFor a SOC analyst needing to triage security alerts from endpoints efficiently, what\u2019s the best EDR software?\u201d) to consistently yield a narrower set of potential solutions from ChatGPT.<\/p> I think ChatGPT might not be better at tailoring a list of solutions to a specific use case because it doesn\u2019t have a deep understanding of most brands. (More on this data in an upcoming note.)<\/p> In each individual response, ChatGPT will, on average, mention only 10 brands.<\/p> There\u2019s quite a range, though \u2013 a minimum of 6 brands per response and a maximum of 15 when averaging across response sets.<\/p> But a single response typically names about<\/strong> 10 brands regardless of category or prompt type.<\/p> The big difference is in how much the pool of brands rotates across responses \u2013 competitive categories draw from a much deeper bench, even though each individual response names a similar count.<\/p> Everything old (in SEO) truly is new again (in GEO\/AEO). It reminds me of trying to get a placement in one of Google\u2019s \u201c10 blue links\u201d.<\/p> Dig deeper: How to measure your AI search brand visibility and prove business impact<\/em><\/strong><\/p> <\/p> Get the newsletter search marketers rely on.<\/p> <\/p><\/div> <\/p><\/div> <\/div> <\/p> When you ask ChatGPT for a B2B software recommendation 100 different times, there are only ~5 brands, on average, that it\u2019ll mention 80%+ of the time.<\/p> To put it in context, that\u2019s just 11% of all the 44 brands it\u2019ll mention at all across those 100 responses.<\/p> So it\u2019s quite competitive to become one of the brands ChatGPT consistently mentions whenever someone asks for recommendations in your category.<\/p> As you\u2019d expect, these \u201cdominant\u201d brands tend to be big, established brands with strong recognition. For example, the dominant brands in the accounting software category are QuickBooks, Xero, Wave, FreshBooks, Zoho, and Sage.<\/p> If you\u2019re not a big brand, you\u2019re better off being in a niche category:<\/p> When you operate in a niche category, not only are you literally competing with fewer companies, but there are also more \u201copen slots\u201d available to you to become a dominant brand in ChatGPT\u2019s responses.<\/p> In niche categories, 21% of all the brands ChatGPT mentions are dominant brands, getting mentioned 80%+ of the time.<\/p> Compare this to just 7% of all brands being dominant in competitive categories, where the majority of brands (72%) are languishing in the long tail, getting mentioned less than 20% of the time.<\/p> A nuanced prompt doesn\u2019t dramatically change the long tail of little-seen brands (with <20% visibility), but it does change the \u201cwinner\u2019s circle.\u201d Adding persona context to a prompt makes it a bit more difficult to reach the dominant tier \u2013 you can see the steeper \u201ccliff\u201d a brand has to climb in the \u201cnuanced prompts\u201d graph above.<\/p> This makes intuitive sense: when someone asks \u201cbest accounting software for a Head of Finance,\u201d ChatGPT has a more specific answer in mind and commits a bit more strongly to fewer top picks.<\/p> Still, it\u2019s worth noting that the overall pool doesn\u2019t shrink much \u2013 ChatGPT mentions ~42 brands in 100 responses to nuanced prompts, just a handful fewer than the ~46 mentioned in response to simple prompts. If nuanced prompts make the winner\u2019s circle a bit more exclusive, why don\u2019t they also narrow the total field?<\/p> Partly, it could be that the \u201cnuanced\u201d questions we fed it weren\u2019t meaningfully more narrow and specific than what was implied in the simple questions we asked.<\/p> But, based on other data I\u2019m seeing, I think this is partly about ChatGPT not knowing enough about most brands to be more selective. I\u2019ll share more on this in an upcoming note.<\/p> Dig deeper: 7 hard truths about measuring AI visibility and GEO performance<\/em><\/strong><\/p> It\u2019s never been more important to differentiate. 21% of mentioned brands reach dominant status in niche categories vs. 7% in competitive ones.<\/p> Without time and a lot of money for brand marketing, an upstart tech company isn\u2019t going to become a dominant brand in a broad, established category like accounting software.<\/p> But the field is less competitive when you lean into your unique, differentiating strengths. ChatGPT is more likely to treat you like a dominant brand if you work to make your product known as \u201cthe best accounting software for commercial real estate companies in North America.\u201d<\/p> Given the inconsistency of ChatGPT\u2019s recommendations, a single spot-check for any given prompt is nearly meaningless. Unfortunately, checking each prompt just once per time period is exactly what most AI visibility tracking tools do.<\/p> If you want anything approaching a statistically-significant visibility score for any given prompt, you need to run the prompt at least dozens of times, even 100+ times, depending on how precise you need the data to be.<\/p> But that\u2019s obviously not practical for most people, so my suggestion is: For the key, bottom-of-funnel prompts you\u2019re tracking, run them each ~5 times whenever you pull data.<\/p> That\u2019ll at least give you a reasonable sense of whether your brand tends to show up most of the time, some of the time, or never.<\/p> Your goal should be to have a confident sense of whether your brand is in the little-seen long tail, the visible middle, or the dominant top-tier for any given prompt. Whether you use my tiers of \u2018under 20%\u2019, \u201820\u201380%\u2019, and \u201880%+\u2019, or your own thresholds, this is the approach that follows the data and common sense.<\/p> See the complete picture<\/span> of your search visibility. <\/p> Track, optimize, and win in Google and AI search from one platform. <\/p> <\/p><\/div> Start Free Trial<\/span> <\/p> Get started with<\/p> In future newsletters and LinkedIn posts, I\u2019m going to build on these findings with new research:<\/p> This article was originally published on Visible on beehiiv (as <\/em>Most AI visibility tracking is misleading (here\u2019s my new data)<\/em>) and is republished with permission.<\/em><\/p> <\/div> Contributing authors are invited to create content for Search Engine Land and are chosen for their expertise and contribution to the search community. Our contributors work under the oversight of the editorial staff and contributions are checked for quality and relevance to our readers. Search Engine Land is owned by Semrush. Contributor was not asked to make any direct or indirect mentions of Semrush. The opinions they express are their own.<\/em> <\/p> Opinion#repeated #ChatGPT #runs #reveal #brand #visibility1770764773<\/p> ","protected":false},"excerpt":{"rendered":" We know AI responses are probabilistic \u2013 if you ask an AI the same question 10 times, you\u2019ll get 10 different responses. But how different are the responses? That\u2019s the question Rand Fishkin explored in some interesting research. And it has big implications for how we should think about tracking AI visibility for brands. In […]<\/p>\n","protected":false},"author":1,"featured_media":3255,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[18],"tags":[405,94,155,10674,4715,9430,76],"class_list":["post-3254","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-careers","tag-brand","tag-chatgpt","tag-opinion","tag-repeated","tag-reveal","tag-runs","tag-visibility"],"acf":[],"_links":{"self":[{"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/posts\/3254","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3254"}],"version-history":[{"count":0,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/posts\/3254\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/media\/3255"}],"wp:attachment":[{"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3254"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3254"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3254"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Methodology<\/h2>
![\"Semrush](\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/semrush-one.webp\" "\\"What")
<\/div> <\/p><\/div> Findings<\/h2>
How many \u2018open slots\u2019 in ChatGPT responses are available to brands?<\/h3>
![\"How](\"https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/02\/How-many-brands-does-ChatGPT-draw-from-on-average.png\" "\\"What")
<\/figure> <\/div> Return of the \u201910 blue links\u2019<\/h3>
![\"How](\"https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/02\/How-many-brands-per-response-on-average.png\" "\\"What")
<\/figure> <\/div>
How consistent are ChatGPT\u2019s brand recommendations?<\/h3>
![\"ChatGPT](\"https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/02\/ChatGPT-knows-44-brands-in-your-category.png\" "\\"What")
<\/figure> <\/div> ![\"It's](\"https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/02\/Its-easier-to-get-good-AI-visibility-in-niche-categories.png\" "\\"What")
<\/figure> <\/div> ![\"The](\"https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/02\/The-responses-to-nuanced-prompts-are-harded-to-dominate.png\" "\\"What")
<\/figure> <\/div> What does this mean for B2B marketers?<\/h2>
If you\u2019re not a dominant brand, pick your battles \u2013 niche down<\/h3>
Most AI visibility tracking tools are grossly misleading<\/h3>
<\/div> <\/p><\/div> What\u2019s next?<\/h2>