Every AI visibility tracker pulls from one of four data sources: the provider’s API, logged-out scraping, logged-in scraping, or paid logged-in panels. Ethan Smith, whose agency Graphite runs Webflow’s answer-engine program, laid out the taxonomy in an interview with Niklas Buschner (source). Vendors compete on dashboards. The source question decides whether the numbers underneath mean anything, and no pricing page mentions it.
Do the four sources return different results?
The answers change little across sources; the citations change a lot. That asymmetry is Smith’s sharpest technical point in the interview. Presence is source-tolerant: whether your brand appears in an answer reads about the same from the API as from a logged-in browser session. The pages the engine cites do not transfer, and citations are the part you act on.
Smith on the API: “what you really care about is probably logged in citations, like nobody’s using the API” (interview). Build an off-site strategy from API-derived citations and, in his words, “it’s going to be way off. You’re going to be optimizing citations that are not used in logged in.”
The practical translation: a tracker can report your mention rate from any of the four sources. The moment it sells you a target list of pages to get cited on, the source decides whether that list is real. A list built from API answers optimizes for a surface none of your buyers use.
Smith once counted 60 answer-tracking tools and told people to pick the cheapest. The interview adds the asterisk: presence tracking is the commodity, citation-level data is where the sources split. His price forecast for the category: “It’s going to cost $80 to $150 eventually because [it’s] a commodity.” The dashboard gets the polish because the dashboard is the part you can see in a demo.
Is panel data ready?
No. Smith’s verdict on paid logged-in panels: “I don’t know what the panel is… I’ll wait until I have some more confidence” (interview). His test for a panel is representativeness, and his analogy comes from polling: 1,000 representative respondents beat 30 million Californians. Panel size on a sales deck answers a different question than panel composition, and composition is the one that matters.
The hole underneath goes deeper than panels. “None of the LLMs give us prompt data,” Smith notes: search has four first-party volume sources (Google Ads, Search Console, Bing impressions, Bing Ads) while the LLMs provide zero. Every prompt-volume figure in every tracker is inference. Smith’s sanity check: search engines get around 25x the page views of LLMs per SimilarWeb data, so expected prompt volume is search volume divided by 25. “The closer prompt volume is to search volume, the more off it is.”
His own fallback when the numbers matter: “If we really care, we’re literally just having humans copy and paste things into spreadsheets” (interview). A category with 60 vendors, and the reference method is a human with a spreadsheet.
What should you demand from any tracker?
Four disclosures, before you look at a single chart:
- The data source, per engine. API, logged-out, logged-in, or panel. If the vendor cannot answer per engine, the answer is API.
- Run counts. Answers are a probability distribution. Graphite’s own study found you need the same prompt 7 to 10 times to get a decent view of that distribution (interview). A number without a run count is one sample dressed as a rate: the math on why single runs lie.
- Citation-level data from consumer surfaces. Mention rates can come from anywhere. Citation recommendations must come from the surface your buyers use.
- Labels on anything API-derived. API data is fine for presence. The report has to say so where the client reads it.
Our own practice, since the list applies to us too: the free scan runs five buyer questions through the Gemini API, and the report labels those results as API-sampled snapshots. The paid tier runs each question 7 times per engine and adds consumer-surface answers pulled through scraping vendors, labeled by source. Every finding links to the captured evidence. Nothing in either report is called a rank.
Ask your current vendor which of the four sources feeds each engine on your dashboard. The answer takes one email and reprices the subscription either way. Or start from a labeled baseline: run the free scan.