Analysis results
Load example data or enter values, then calculate the curve.
Automatic interpretation
Experiment summary
- Dataset
- —
- Model
- —
- Metric
- —
- Selection method
- —
- Cross-validation folds
- —
- Sample size
- —
- Random seed
- —
- Confidence level
- —
Detailed results
| Complexity | Training | Validation | Test | Gap | Score change | Marginal improvement | Std. dev. | Confidence interval | Region | Optimal |
|---|---|---|---|---|---|---|---|---|---|---|
| No results yet. | ||||||||||
Formula used
For higher-is-better metrics, a large positive gap can indicate overfitting. For lower-is-better error metrics, the calculator compares validation error against training error. It also reverses improvement signs where required.
The one-standard-error rule selects the simplest model within one standard error of the best validation score. The balanced method combines normalized validation quality with a complexity penalty. Elbow detection finds the strongest change in slope.
How to use
- Choose the experiment, model, metric, and complexity measure.
- Enter training, validation, and optional test values.
- Add deviations when cross-validation uncertainty is available.
- Select an optimal-complexity method and smoothing option.
- Adjust chart labels, scales, markers, regions, and thresholds.
- Calculate, interpret the result, and export the analysis.
Example data
| Polynomial degree | Training RMSE | Validation RMSE |
|---|---|---|
| 1 | 8.8 | 9.3 |
| 2 | 6.4 | 6.9 |
| 3 | 4.9 | 5.4 |
| 4 | 3.8 | 4.8 |
| 5 | 3.0 | 5.2 |
| 6 | 2.4 | 6.1 |
Understanding model complexity
Bias and variance
Simple models often have high bias and stable predictions. Complex models can reduce bias but increase variance. The preferred model balances both effects.
Underfitting
Underfitting occurs when training and validation performance are both weak. Increasing useful complexity may improve both curves. More informative features can also help.
Overfitting
Overfitting appears when training performance keeps improving while validation performance worsens. The widening gap signals reduced generalisation. Regularisation or simpler models may help.
Reading the curve
Follow validation performance across ordered complexity values. Identify the best region, then inspect uncertainty and gaps. Prefer simpler models when performance is nearly equal.
Frequently asked questions
What does model complexity mean?
It measures a model’s flexibility or capacity. Examples include tree depth, polynomial degree, estimator count, or parameter count.
Which validation metric should I use?
Use the metric matching your prediction goal. Classification and regression tasks usually require different measures.
Why can training and validation curves diverge?
A complex model may memorize training patterns. Its performance on unseen validation data can then decline.
What is the one-standard-error rule?
It selects the simplest model whose validation result remains within one standard error of the best model.
Should I smooth the curves?
Smoothing can reveal broad trends. Always retain original values so important variation remains visible.
Can I use logarithmic axes?
Yes, when all plotted values are positive. The calculator warns about zero or negative values.
How many complexity points are required?
At least three points are recommended. More points usually produce a clearer trend and better elbow estimate.
What does a small generalisation gap mean?
It suggests similar training and validation behavior. However, both performances may still be poor.
Can I compare different algorithms?
Yes. Save each run and overlay validation curves. Comparable metrics and scales produce the clearest comparison.