September 15, 2026
8 min read
Measuring search relevance: the short answer
Measure ecommerce search relevance with two kinds of evidence: judged result quality and shopper outcomes. NDCG and MRR compare results with a labeled test set. Zero-result rate, search conversion rate, add-to-cart rate, and revenue per search show what shoppers do. Use both so a metric does not hide a search problem.
What the main metrics mean
| Metric | Definition | Use it for |
|---|---|---|
| Zero-result rate | Searches returning no products | Finding coverage and catalog gaps |
| MRR | How high the first relevant result appears | Known-item and navigational queries |
| NDCG | Relevance weighted by result position | Comparing judged result pages |
| Search conversion | Orders after a search | Connecting discovery with purchase |
| Revenue per search | Revenue divided by searches | Comparing commercial query value |
Build a useful test set
Use real high-volume, high-revenue, zero-result, and recently changed queries. Label the first page for exact match, product type, required attributes, availability, and overall usefulness. Keep the labels consistent and record why a result received its score.
Segment the set by query intent. A known SKU should not be judged like a broad category query. A use-case query may have several acceptable products, while an exact brand query may have a narrow expected result.
Pair quality with behavior
Run a baseline before changing ranking, synonyms, or facets. Compare the same query groups after the change. Watch conversion, add-to-cart, exits, and revenue per search by device and category. Review outliers manually because averages can hide a bad result page for a valuable query.
Clerk.io’s Intelligent Search includes search analytics. Hairlust reported €21,000 from Search and Recommendations in the last 30 days, showing why search should be evaluated against commercial outcomes as well as technical scores.
Free ebook: Clerk.io’s Search covers search conversion and measurement.
Start a free trial or book a demo to build a measurement plan.
TL;DR
Use NDCG and MRR for judged result quality. Use zero-result rate, conversion, add-to-cart rate, exits, and revenue per search for shopper and business outcomes. Segment by query intent and inspect important queries manually.
