Non-parametric memory

Non-parametric memory is the knowledge an AI model looks up while it answers, such as web pages it searches for, as opposed to what it learned in training.

Non-parametric memory is the knowledge an AI model looks up while it answers, rather than knowledge it learned in training. When ChatGPT or Perplexity searches the web, reads a few pages and writes an answer from them, those pages are its non-parametric memory for that answer.

This is the part of an AI answer you can change quickly. What a model learned in training stays fixed until the next model comes out. What it looks up can change as soon as the pages it reads change.

Where the term comes from

The term comes from the 2020 research paper that introduced retrieval-augmented generation, or RAG. The authors split a model's knowledge in two. What's stored in the model's own settings, its parameters, is parametric memory. What it fetches from an outside collection of documents at the time of the question is non-parametric memory.

The idea is simple even if the name isn't: the model has things it knows, and things it looks up.

Why it matters to you

Most AI platforms now look things up for many questions, especially ones about products, prices and recent events. When they do, the pages they read shape what they say, and those pages are often reviews, list articles, directories and forum threads.

For example, say someone asks Perplexity which roasters deliver across the UK. Perplexity searches, reads three articles, and names the roasters those articles name. If one of the articles says Fernhill Coffee only delivers within London, Perplexity may repeat it, whatever the model learned in training. Get that page corrected, and the next answer can be right.

So working on the pages the platforms read is the faster route. Changing what a model learned in training is slow and indirect.