Does ChatGPT Even Mention You?

You can rank #1 on Google for a keyword and still be invisible for the question that actually matters, because increasingly that question never touches a search-results page. It goes to ChatGPT, Claude, or Perplexity, and the answer either names your product or it doesn't — there's no ten-blue-links fallback where you show up anyway. On a product we built, we stopped guessing about that and started measuring it: a small monitor that asks the answer engines the questions our buyers actually ask, logs whether we get cited, and sits next to the classic Search Console numbers so we can see both funnels moving at once, or not moving, in the same place.

This is a write-up of that monitor: why "we rank well" and "the AI recommends us" are two different, only loosely correlated facts, how to check the second one programmatically instead of by hand-testing prompts, why the Search Console side still matters and has its own quirks, and where the method's honest limits are — because there is no citation-rank API, and anyone who tells you otherwise is selling something.

§ 01Two funnels, not one metric with a new name

Classic SEO answers one question: for a given query, where does your URL land in a ranked list a human then chooses from. Search-console monitoring is built entirely around that model — position, impressions, click-through rate, all indexed to a rank. Answer-engine optimization answers a structurally different question: when an assistant synthesizes one answer instead of returning a list, does your product get named inside it. There's no position 4 to climb to. You're either a source the model drew on, or you aren't in the answer at all — and the user, in a lot of these flows, never sees a URL to click regardless.

Those two funnels correlate, but loosely. Good indexed content is a precondition for both — Perplexity in particular retrieves from the live web before it answers, so if you're not crawlable and indexed you're structurally excluded from the pool it can even consider citing. But being retrievable and being selected are different filters. Perplexity's documented retrieval pipeline pulls somewhere around 5–10 candidate pages per query and passes them through relevance, recency, and authority reranking before a synthesis stage decides which ones actually get quoted — of that pool, typically only 3–4 survive to citation. You can win the first filter and lose the second. That's the case for treating AEO as its own thing to track, not a subheading under SEO.

§ 02Defining "cited" so a script can check it

Before writing anything that queries an API, you need a falsifiable definition of a citation — otherwise you're eyeballing chat transcripts forever. Ours is deliberately narrow: send the assistant a natural question a real prospect would ask, phrased as a person would phrase it, not as a keyword. Take the response text plus any source list the API returns, lowercase the whole thing, and check it for your brand terms — product name, close variants, maybe a founder's name if that's how people search for you. A hit is a citation. No hit is not, even if the answer is topically perfect and simply names three competitors instead.

// the citation check, provider-agnostic: does the answer name us?
function brandsIn(text: string, sources: string[], brands: string[]): string[] {
  const hay = `${text}\n${sources.join("\n")}`.toLowerCase();
  return brands.filter(b => hay.includes(b.toLowerCase()));
}

const PROMPT = (q: string) =>
  `A user asks: "${q}". Answer helpfully and name specific
   apps/tools/resources you would recommend.`;

The prompt matters more than it looks. Ask a leading question — "is Product X good for Y" — and you'll get a flattering, useless answer because you told the model what to talk about. The queries have to be the ones a stranger would type, sourced the same way you'd source SEO target keywords: real phrasing, not internal jargon. We keep that query list in a plain text file, one per line, versioned like any other config — because the list itself is a product decision, not an implementation detail, and it changes as often as the product's positioning does.

§ 03Querying three engines the same way, honestly

Anthropic, OpenAI, and Perplexity each expose the underlying call as a plain chat-completion request, so the same prompt and the same brand-detection function run against all three with only the transport differing. Anthropic and OpenAI's chat endpoints don't return a source list at all — for those two, "cited" can only mean the brand appears in the generated text itself, because there's nothing else to check. Perplexity's sonar models do return a citations array of URLs alongside the answer, since it's doing live retrieval — so for Perplexity, "cited" can mean either the brand name appears in the prose or your domain shows up in that source list, which is a meaningfully stronger signal because it's closer to what the model actually treated as a reference rather than something it merely mentioned in passing.

// each provider check returns null if unconfigured, else a normalized result
async function checkPerplexity(query: string, brands: string[]) {
  const res = await fetch("https://api.perplexity.ai/chat/completions", {
    method: "POST",
    headers: { authorization: `Bearer ${apiKey}`, "content-type": "application/json" },
    body: JSON.stringify({ model: "sonar", messages: [{ role: "user", content: PROMPT(query) }] }),
  });
  const data = await res.json();
  const text = data.choices?.[0]?.message?.content ?? "";
  const sources = data.citations ?? [];
  return { provider: "perplexity", query, cited: brandsIn(text, sources, brands).length > 0, sources };
}

Two decisions worth calling out because they're easy to get wrong. First: only query providers you've actually configured a key for, and skip the rest silently — a monitor that throws on a missing OPENAI_API_KEY is a monitor nobody runs reliably. Second: log the raw snippet and source list alongside the boolean, not just the boolean. The pass/fail number is what you alert on, but the text is what tells you why — whether you're losing citations to a specific competitor, being described accurately but by the wrong name, or simply absent from a category the model clearly knows exists.

§ 04Tracking change, not just state

A single "cited: true/false" snapshot answers "are we visible today," which is interesting exactly once. The useful signal is the transition — newly cited, or newly dropped — because that's the thing worth a Slack ping and the thing that correlates with something you actually did: shipped a piece of content, got a mention on a review site, changed positioning copy. Every run stores its result and looks up the immediately-preceding result for the same query and provider before deciding whether to alert.

// compare this run against the last one, per query+provider — alert on flips only
for (const query of queries) {
  const results = await checkQuery(query);
  await saveAeo(db, results);
  for (const r of results) {
    const before = await priorAeo(db, query, r.provider);
    if (r.cited && !before?.cited) alert(`newly cited by ${r.provider} for "${query}"`);
    if (!r.cited && before?.cited) alert(`dropped by ${r.provider} for "${query}"`);
  }
}

This is also where you earn the right to say something concrete about causality later. "We published the comparison page and three weeks later Perplexity started citing us for the category query" is a claim you can only make if you were storing results the whole time — not one you can reconstruct after the fact from memory or from a single spot-check someone ran the day a stakeholder asked.

§ 05The Search Console half — same discipline, different API

None of this replaces watching classic search, it runs beside it. The Search Console searchAnalytics.query endpoint gives you clicks, impressions, CTR, and average position per query, authenticated with a read-only service account rather than a personal login — which matters if this is going to run on a schedule instead of a human opening a dashboard once a month. One quirk worth planning around from the start: GSC's own documentation puts the reporting lag at roughly two to three days, so a query for "yesterday" returns nothing useful — every fetch has to date-offset the window back by a few days or it silently returns thin, unfinalized rows.

// GSC fetch, offset for the reporting lag before building the date window
const end = new Date(); end.setUTCDate(end.getUTCDate() - 3); // GSC lag
const start = new Date(end); start.setUTCDate(start.getUTCDate() - 28);

const res = await fetch(`${searchConsoleUrl}/sites/${site}/searchAnalytics/query`, {
  method: "POST",
  headers: { authorization: `Bearer ${token}` },
  body: JSON.stringify({ startDate: ymd(start), endDate: ymd(end), dimensions: ["query"], rowLimit: 1000 }),
});

The report worth generating from that isn't a raw table, it's a diff: pull the latest snapshot and the one from roughly seven days earlier, join them on query, and surface two lists — your named target keywords regardless of movement, and anything else that moved more than a few positions in either direction. Flag the targets so they're always visible even on a quiet week, and cap the movers list so a noisy week doesn't bury the report in three-position wobbles that mean nothing.

§ 06Getting indexed faster, and where the method actually ends

Both halves depend on being crawled promptly, so shipping content and waiting for the next organic crawl is a real cost. IndexNow is a small push protocol, jointly backed by Bing and Yandex, that lets you notify participating engines the moment a page changes instead of waiting for their scheduler — you host a key file, then POST the changed URLs to a single endpoint. It's worth wiring in for exactly one reason: faster indexing shortens the gap between "we shipped the page" and "the page is even eligible to be retrieved," which gates both the GSC numbers and, for retrieval-based engines like Perplexity, the AEO numbers too. Note the asterisk: Google isn't part of IndexNow — that side still goes through the standard sitemap submission in Search Console.

Now the limits, stated plainly rather than buried in a footnote. There is no official "AI citation rank" API from any of the three providers — this whole approach is a proxy, built by asking the same question the same way and watching the answer change, not a number any vendor publishes or guarantees stays stable. The checks are also not perfectly reproducible: independent benchmarking of production LLM APIs has repeatedly found that even a temperature of zero doesn't guarantee identical output run to run, because batching and inference-serving conditions on the provider's side introduce variance you don't control. A single "not cited" run on a single day is noise; a query that goes uncited across several consecutive weekly runs is signal. Size your alerting threshold accordingly — we alert on a flip, but we don't treat one flip as gospel until it holds for more than one cycle.

The absence of a ranking API isn't a reason to skip AEO monitoring — it's the reason to store every raw answer, not just the boolean.

Put together, the two halves answer two different stakeholder questions with the same weekly cadence: "are we still findable in classic search" from the GSC deltas, and "does the AI actually recommend us when someone asks" from the AEO checks. Neither one alone tells you whether the product is actually visible where its buyers are looking now — you need both dashboards open, not one relabeled as the other. We built the first version of this into the language learning app specifically because AEO needed no live, indexed domain to start producing signal on day one, while the GSC half switched on once the site had something to be indexed.

— End of essay. Want to know if the answer engines actually recommend your product? Start a project →