Formula Used
Higher-is-better metric: Generalisation gap = training score − validation score.
Lower-is-better metric: Generalisation gap = validation error − training error.
Gap percentage: absolute gap ÷ absolute training metric × 100.
Cross-validation variation: standard deviation ÷ fold mean × 100.
How to Use
- Select the metric and choose its preferred direction.
- Enter training, validation, and optional test performance.
- Set acceptable, warning, and severe gap thresholds.
- Paste epoch curves or build the editable epoch table.
- Add cross-validation folds and optional model context.
- Submit the form and review the complete diagnosis.
Example Data
| Scenario | Training | Validation | Likely interpretation |
|---|---|---|---|
| Balanced classifier | 0.88 | 0.86 | Small gap and good generalisation |
| Mild overfitting | 0.93 | 0.86 | Moderate performance separation |
| Severe overfitting | 0.99 | 0.73 | Large training advantage |
| Underfitting | 0.62 | 0.60 | Both scores remain weak |
Understanding the Results
What Overfitting Means
Overfitting occurs when a model memorises training patterns. Validation performance then fails to improve consistently. New-data predictions become less reliable and less stable.
Overfitting Versus Underfitting
Overfit models score strongly on training data. Underfit models perform poorly across every dataset. Balanced models maintain similar, useful performance across splits.
Why Learning Curves Matter
Learning curves show changes throughout model training. Diverging curves often reveal delayed validation deterioration. The best epoch usually appears before divergence accelerates.
Important Limitation
No single threshold proves that overfitting exists. Dataset noise and task difficulty influence every gap. Confirm findings using repeated validation and domain evidence.
Frequently Asked Questions
What is a good generalisation gap?
A smaller gap usually indicates better generalisation. Acceptable values depend on metric scale and risk. Compare gaps across consistent experiments and identical splits.
Can validation outperform training?
Yes, regularisation and augmentation can lower training performance. Small differences may be completely normal. Large differences still deserve a leakage review.
Does a high training score prove overfitting?
No, strong training performance alone proves very little. Validation and test behaviour provide essential context. Learning curves provide stronger evidence than one score.
Why include cross-validation?
Cross-validation tests performance across several data partitions. Stable folds increase confidence in generalisation estimates. Unstable folds reveal split sensitivity and sampling problems.
Which metric should I choose?
Choose the metric matching your practical objective. Accuracy may mislead on imbalanced classification tasks. Error metrics often suit continuous prediction problems better.
How does early stopping help?
Early stopping preserves the best validation checkpoint. It prevents unnecessary fitting after progress stops. Patience allows small temporary fluctuations during training.
Can more data reduce overfitting?
More representative data often improves model generalisation. Quality matters more than raw sample quantity. Duplicates and leakage can worsen misleading confidence.
Should I reduce model complexity?
Complexity reduction can lower model variance substantially. Remove layers, parameters, features, or interactions carefully. Retest performance after every controlled model change.
Can this calculator replace evaluation experiments?
No, this calculator provides structured diagnostic guidance. It cannot inspect every dataset-specific failure mechanism. Use it alongside robust experimental validation practices.