Threshold Performance Plot Calculator

Formula Used

Accuracy = (TP + TN) / (TP + TN + FP + FN)
Precision = TP / (TP + FP)
Recall = TP / (TP + FN)
Specificity = TN / (TN + FP)
F1 = 2 × Precision × Recall / (Precision + Recall)
Fβ = (1 + β²) × Precision × Recall / (β² × Precision + Recall)
Balanced Accuracy = (Recall + Specificity) / 2
Youden's J = Recall + Specificity − 1
Expected Cost = FP × CostFP + FN × CostFN
Expected Utility = TP × BenefitTP + TN × BenefitTN − Expected Cost

How to Use

Paste labels and predicted probabilities into the data box. Parse the data, then map each required column. Choose one or several model probability columns.

Set threshold ranges, metrics, constraints, costs, and plot preferences. Select an objective that matches your deployment priorities. Run the analysis and inspect all results.

Use the threshold slider to examine confusion-matrix changes. Export tables, charts, summaries, or printable PDF reports. Validate chosen thresholds using unseen production-like data.

Example Data

Interpretation and Limitations

Lower thresholds usually increase recall and predicted positive counts. Higher thresholds often improve precision while reducing recall. Error costs should guide the final cutoff.

Accuracy may mislead when classes are severely imbalanced. Calibration quality affects every probability-based threshold decision. Always evaluate thresholds on representative validation data.

Bootstrap intervals describe sampling variation, not future distribution shifts. Small datasets can create unstable optimal threshold estimates. Production monitoring remains essential after threshold deployment.

Frequently Asked Questions

What does a classification threshold control?

It converts predicted probabilities into class decisions. Values above the cutoff become positive predictions. Different cutoffs change every confusion-matrix based metric.

Is 0.50 always the best threshold?

No, 0.50 is only a common default. The best value depends on objectives and costs. Validation evidence should determine the deployed decision threshold.

Why does recall rise at lower thresholds?

More observations receive positive classifications at lower cutoffs. This captures additional true positives and false positives. Recall rises unless no positive cases remain missed.

Which metric suits imbalanced datasets?

F1, balanced accuracy, MCC, and precision-recall measures help. Each metric emphasizes different classification behavior and risks. Choose according to operational consequences and prevalence.

What is Youden's J statistic?

It combines sensitivity and specificity into one score. The statistic equals recall plus specificity minus one. Larger values indicate stronger balanced discrimination performance.

How does F-beta differ from F1?

F-beta changes the relative emphasis on recall. Beta above one weights recall more heavily. Beta below one gives precision greater importance.

Why use cost-sensitive optimization?

False positives and false negatives rarely cost equally. Cost settings translate errors into practical consequences. The chosen threshold then minimizes expected operational loss.

What are bootstrap confidence bands?

Bootstrap samples repeatedly resample the supplied observations. The resulting metric distributions estimate sampling uncertainty. More samples improve stability but require computation.

Can several models be compared?

Yes, select multiple probability columns before running analysis. Each model receives separate curves and optimal thresholds. Optional split columns compare validation and test behavior.

How should tied thresholds be handled?

Tied objectives may produce several equally strong cutoffs. Prefer a stable region rather than one point. Operational simplicity can break otherwise equivalent threshold ties.