Inference
Inference is running a trained AI model to get an output. Every answer an AI platform gives you is one inference.
Inference is when a trained AI model is run to produce something, like an answer. Training is when the model learns. Inference is when it's used. Every time someone asks ChatGPT a question and gets an answer back, that's an inference.
You'll mostly hear the word when people talk about speed and cost. Each answer costs the platform computing power, and that shapes which model you get.
Why it matters to a marketer
Running a big model is expensive. So platforms don't always use their biggest model for every question. They may route simple questions, or people who aren't signed in or paying, to a smaller and faster one.
That means the answer your customer sees can come from a different model than the one in the news, and different models can name different brands. For example, someone on a free plan who asks "which coffee grinder should I buy?" might get a quick answer from a small model that names two brands, while a paying user gets a longer answer from a bigger one that names five.
Inference and the web
Inference doesn't have to rely only on what the model learned in training. Many platforms search the web first and give the model the pages they found, then run the inference. That's how an answer can mention a product that launched after the model was trained.
In AnswerOrbit
We ask each platform in its own app, as someone who isn't signed in, and the platform decides which model answers, the same as it would for anyone. Each answer shows the model that gave it. See How we count and Answers.