Results
Run the calculator to see results.
| Bin | Count | Weight | Mean forecast | Event rate | Reliability part |
|---|
| Group | Rows | Weight | Brier score | Mean probability | Event rate |
|---|
| Rank | Model | Brier score | Skill score | Difference from best | Status |
|---|
Formula used
Binary: BS = Σ wᵢ(pᵢ − yᵢ)² / Σ wᵢ
Multiclass: BS = Σᵢ Σₖ wᵢ(pᵢₖ − yᵢₖ)² / Σᵢ wᵢ
Skill score: BSS = 1 − BSmodel / BSreference
Binary decomposition: BS = Reliability − Resolution + Uncertainty
How to use
- Select binary or multiclass prediction mode.
- Enter probabilities, outcomes, and optional row information.
- Choose calibration, baseline, confidence, and filtering options.
- Add comparison models when model ranking is required.
- Calculate, inspect charts, then export the complete report.
Example data
| Probability | Actual | Weight | Group | Date |
|---|---|---|---|---|
| 0.90 | 1 | 1 | High confidence | 2026-07-01 |
| 0.70 | 1 | 1 | Medium confidence | 2026-07-02 |
| 0.40 | 0 | 1 | Medium confidence | 2026-07-03 |
| 0.20 | 0 | 1 | Low confidence | 2026-07-04 |
Understanding the Brier score
The Brier score evaluates the quality of probability forecasts. Lower values represent smaller squared prediction errors. A score of zero indicates perfect probabilities.
Accuracy evaluates final class decisions, not probability quality. Calibration compares predicted probabilities with observed event frequencies. Resolution rewards useful separation between different risk levels.
Class imbalance can make an unskilled score appear deceptively small. Compare each model against an appropriate reference forecast. Skill scores summarize improvement over that selected baseline.
Log loss penalizes confident mistakes more severely than Brier loss. Brier loss remains bounded for binary outcomes. Both metrics should support calibration and discrimination analysis.
Frequently asked questions
What is a good Brier score?
Lower scores are better. Compare against prevalence and baseline forecasts.
What is the binary score range?
Binary Brier scores normally range from zero through one.
Why can multiclass ranges differ?
Some definitions divide the summed class error by two.
What does a positive skill score mean?
The tested model improves upon the selected reference forecast.
Can observation weights be used?
Yes. Weighted errors are divided by total valid weight.
What does reliability measure?
Reliability measures differences between predicted and observed bin rates.
What does resolution measure?
Resolution measures separation from the overall observed event rate.
How are confidence intervals calculated?
Bootstrap resampling estimates uncertainty around the reported Brier score.
Can multiple models be compared?
Yes. Add aligned probabilities for every alternative model.