Formula used
MRR = (1 / Q) × Σ(1 / rankᵢ)Each query contributes the reciprocal of its first relevant rank. A first-place result contributes one. A missing result normally contributes zero.
MRR@K sets contributions beyond rank K to zero. Standard MRR gives each query equal importance. Weighted MRR changes that convention.
How to use the calculator
- Select direct ranks, relevance lists, table input, or CSV.
- Enter Model A data and optionally enable Model B comparison.
- Choose the missing-result policy, cutoff, weighting, and precision.
- Submit the form to view MRR, diagnostics, tables, and charts.
- Copy results or export the detailed table as CSV or PDF.
Example data
| Query | First relevant rank | Reciprocal rank |
|---|---|---|
| Q1 | 1 | 1.0000 |
| Q2 | 3 | 0.3333 |
| Q3 | 2 | 0.5000 |
| Q4 | None | 0.0000 |
The example MRR is (1 + 1/3 + 1/2 + 0) ÷ 4 = 0.4583.
Interpretation and limitations
MRR ranges from zero to one. Higher values indicate earlier first relevant results. A value of one means every query succeeds first.
MRR ignores relevant documents after the first relevant result. Use MAP or NDCG when later relevant items matter. Use Recall@K for coverage.
Comparisons remain meaningful only with identical query sets and relevance rules. Report cutoffs and missing-result policies clearly. Weighted results require additional explanation.
Frequently asked questions
What does MRR measure?
It measures how early the first relevant result appears across queries.
What is a good MRR value?
Higher is better, but useful benchmarks depend on the task and dataset.
How are missing relevant results handled?
Standard practice gives them zero contribution while keeping them in the denominator.
What is MRR@K?
It treats first relevant results appearing after rank K as unsuccessful.
Can MRR exceed one?
No. Standard and properly normalised weighted MRR remain between zero and one.
How does MRR differ from MAP?
MRR uses only the first relevant result. MAP considers all relevant results.
How does MRR differ from NDCG?
NDCG supports graded relevance and evaluates multiple ranked positions.
Should queries have different weights?
Standard MRR uses equal weights. Custom weights require transparent justification.
Where is MRR commonly used?
It is common in search, question answering, recommendation, and retrieval evaluation.