ROUGE Result Summary
Metric Comparison
Length Comparison
Candidate Ranking
| Rank | Candidate | Average F1 | Difference from best | Preview |
|---|
Detailed Scores
| Candidate | Reference | Metric | Precision | Recall | F1 | Matches | Candidate units | Reference units |
|---|
Candidate Highlight
Reference Highlight
Matched Units and Calculation Steps
Formula Used
Precision equals matched units divided by candidate units. Recall equals matched units divided by reference units. Fβ balances both values using the selected beta.
Precision = overlap / candidate_units Recall = overlap / reference_units Fβ = (1 + β²) × Precision × Recall / (β² × Precision + Recall)
ROUGE-N counts overlapping n-grams. ROUGE-L uses longest common subsequences. ROUGE-S uses ordered skip-bigram pairs.
How to Use
- Paste generated text into the candidate field.
- Paste one or more trusted reference summaries.
- Select metrics and preprocessing settings.
- Choose reference and averaging modes.
- Submit the form and inspect each score.
- Export results using CSV, JSON, or PDF.
Example Data
| Candidate | Reference | Expected observation |
|---|---|---|
| The cat sat on the mat. | The cat is sitting on the mat. | Strong unigram overlap. |
| Solar power lowers household costs. | Household solar energy can reduce electricity costs. | Moderate lexical overlap. |
| Markets rose after inflation slowed. | Stocks climbed when inflation data weakened. | Meaning similarity may exceed lexical overlap. |
Metric Guidance and Limitations
ROUGE-1 measures word overlap. ROUGE-2 captures short phrase overlap. ROUGE-L rewards shared ordering without fixed n-grams.
BLEU emphasises precision more heavily. METEOR can include linguistic matching. BERTScore measures contextual similarity using embeddings.
This calculator uses transparent browser implementations. Production libraries may tokenize text differently. Report settings beside every published score.
Frequently Asked Questions
What does a ROUGE score measure?
It measures lexical overlap between generated and reference text.
Is a higher score always better?
Usually, but quality still requires factual and human evaluation.
Which metric suits short summaries?
ROUGE-1, ROUGE-2, and ROUGE-L provide useful complementary views.
How are multiple references handled?
Choose best, average, or individual reference scoring.
What is macro averaging?
It averages completed example scores with equal example weight.
What is micro averaging?
It aggregates overlap and unit totals before scoring.
Should stop words be removed?
Only when your evaluation protocol explicitly requires removal.
Why can implementations disagree?
Tokenization, stemming, sentence splitting, and aggregation may differ.
Does ROUGE detect hallucinations?
No. It cannot reliably verify truth or source faithfulness.
Can I evaluate several model outputs?
Yes. Separate candidates using a standalone triple-pipe delimiter.