Retrieval quality — relevance, grounding, freshness

Retrieval is the component that decides what the model gets to know. It is also the component most teams ship without a single number attached to it. They can tell you the model version, the prompt template, the temperature, and the latency. Ask what fraction of the time the retriever surfaces the document that actually holds the answer, and the room goes quiet.

My position is that retrieval is a product surface with service-level objectives, and that it fails along three distinct dimensions that need to be measured and engineered separately. Relevance: did we find the right material? Grounding: did the answer actually come from what we found? Freshness: is what we found still true? A system can be excellent on one and broken on another, and a single "accuracy" number hides which.

This piece is about how I would measure each dimension, the mechanisms that move each one, and why the evaluation set has to exist before anything gets tuned.