Round setup
Calibration arena
Diagnose the model
Calibration method laboratory
No calibration leaves the model probabilities unchanged.
Feedback
Interactive calibration graphs
Dataset and evaluation controls
Live calibration scorecard
| Metric | Initial | Current | Change | Interpretation |
|---|
ECE formula
ECE = Σ (nb/N) × |observed frequency − average confidence|
Brier score formula
Brier = (1/N) × Σ (pi − yi)²
Learning centre
Calibration versus accuracy
A model may classify well but assign unreliable probabilities. Threshold metrics and probability reliability answer different questions.
Overconfidence
Predictions are too extreme. A 90% bucket may succeed only 70% of the time.
Underconfidence
Predictions stay too close to 50%. Observed outcomes are more decisive than reported confidence.
Sample uncertainty
Small bins can differ from the diagonal by chance. Confidence intervals show that uncertainty.
Temperature scaling
A single temperature softens or sharpens logits. It preserves ranking but may not fix local errors.
Platt scaling
A logistic mapping learns a slope and intercept. It can correct confidence and systematic bias.
Isotonic regression
A flexible monotonic mapping fits complex curves. It needs enough validation data to avoid overfitting.
Histogram binning
Each confidence range maps to its observed rate. Results depend strongly on bin definitions.