Calibration Curve Calculator

Calculator options

Problem type

Positive label

Decimal precision

Actual labels

Use commas, spaces, tabs, or line breaks.

Model 1 probabilities

Every probability must be between zero and one.

Model 2 probabilities

Model 3 probabilities

Model 1 name

Model 2 name

Model 3 name

Observation weights

Group labels

Upload CSV or TSV

Select a file, then use the column-name mappings below.

CSV column mapping

Actual

Probability

Model 2

Model 3

Weight

Group

Binning method

Number of bins

Minimum observations per bin

Custom boundaries

Show empty bins

Merge sparse bins

Show counts on hover

Interval method

Confidence level

Bootstrap iterations

Random seed

Show confidence bands

Recalibration method

Calibration split

Histogram bins

Compare original and recalibrated probabilities

Stratify calibration split

Recalibration uses an internal calibration/evaluation split. Use an independent holdout set for production decisions.

Chart title

X-axis title

Y-axis title

Marker size

Line width

Chart height

Probability format

Chart theme

Show grid lines

Show perfect line

Show histogram

Missing values

Duplicate rows

Invalid probabilities

Minimum confidence filter

Maximum confidence filter

Use observation weights

Include zero and one endpoints

Save settings in browser

Formula used

ECE = Σ (nₘ / n) × |observed rateₘ − mean probabilityₘ|
Brier score = (1 / n) × Σ (probabilityᵢ − outcomeᵢ)²
Log loss = −(1 / n) × Σ [yᵢ log(pᵢ) + (1 − yᵢ) log(1 − pᵢ)]

How to use

Enter actual labels and predicted probabilities.
Select binning, interval, recalibration, and chart options.
Click Calculate calibration to generate metrics and graphs.
Review the diagonal gap, bin errors, and probability distribution.
Export tables, observations, images, or a printable report.

Example data

Observation	Actual	Predicted probability
1	0	0.08
2	0	0.17
3	1	0.28
4	0	0.35
5	1	0.46

Calibration guidance

Calibration measures probability reliability. A calibrated model predicts frequencies that match observed outcomes. It does not measure ranking quality alone.

Points below the diagonal indicate overconfidence. Points above the diagonal indicate underconfidence. Sparse bins require cautious interpretation.

Calibration should use unseen data. Recalibration and evaluation should use separate samples. This avoids optimistic reliability estimates.

Frequently asked questions

What does a calibration curve show?

It compares predicted probabilities with observed event frequencies across probability bins.

What is perfect calibration?

Perfect calibration places the model curve directly on the diagonal reference line.

What does overconfidence mean?

Predicted probabilities are higher than the observed positive rate in corresponding bins.

What does underconfidence mean?

Observed positive rates exceed the model’s average predicted probabilities.

How many bins should I use?

Use enough bins for detail while retaining sufficient observations within every bin.

What is ECE?

Expected Calibration Error summarizes weighted absolute gaps between confidence and observed frequency.

Is a lower Brier score better?

Yes. Lower values indicate more accurate probability predictions for binary outcomes.

Does good AUC guarantee calibration?

No. Discrimination and probability calibration measure different properties of a model.

When should recalibration be applied?

Apply recalibration when probabilities are systematically biased on representative unseen validation data.

Calibration Curve Calculator

Calibration results