Skip to content

API answers are not what people see in ChatGPT

Why measuring an AI tool through its API can tell a different story from the product your customers use.

Two surfaces, two answers

A model reached through its API is not the same thing as the consumer product wrapped around it. The product adds search, memory, location and its own instructions.

A study by Surfer found only 21 to 32 percent brand overlap between API answers and what the consumer interface showed. Practitioners have also shown location changing the answer.

What that means for a report

If you are telling a client what ChatGPT says about them, the figure should come from the surface their buyers use, and the report should say which surface it was.

Slynnk labels every target. ChatGPT, Gemini and Google are collected from their consumer surfaces; an API target is its own line and is never averaged with the scraped one.

Sources

Put it into practice.

Track your brand with the margin of error on every number.