Calculator Settings
Validation Configuration
Models and Learning-Curve Data
| Training size | Training score(s) | Validation score(s) | Action |
|---|
Enter repeated scores with commas or semicolons. Example: 0.81; 0.82; 0.80.
Formula Used
Generalisation gap = Training mean − Validation mean Sample mean = Σx / n Sample standard deviation = √[Σ(x − x̄)² / (n − 1)] Standard error = s / √n Confidence interval = x̄ ± z × standard error Improvement per sample = Δ validation score / Δ training size
For lower-is-better metrics, the calculator reverses ranking and improvement logic. This keeps diagnoses consistent across errors and scores. Forecasts remain estimates rather than guarantees.
How to Use
- Select the metric and its preferred score direction.
- Enter training sizes with training and validation scores.
- Use separators when repeated validation results are available.
- Set targets, confidence level, and a fitting method.
- Add more models when direct comparison is needed.
- Calculate, review diagnoses, then export the results.
Example Data
| Training size | Training score | Validation score |
|---|---|---|
| 100 | 0.96 | 0.71 |
| 250 | 0.92 | 0.77 |
| 500 | 0.89 | 0.81 |
| 1000 | 0.87 | 0.83 |
| 2000 | 0.86 | 0.845 |
This pattern shows falling training performance and rising validation performance. The shrinking gap suggests improving generalisation. Additional data may still provide smaller gains.
Interpretation Guide
High bias Training and validation scores remain weak and close. More data may not solve the issue. A more expressive model may help.
High variance Training performance is strong but validation performance is weaker. More representative data may reduce variance. Regularisation may also help.
Healthy convergence Validation performance improves while the gap narrows. The model is using added data effectively. Gains may eventually plateau.
Unstable validation Confidence intervals or repeated scores vary substantially. Review sampling, leakage, class balance, and noise. More reliable validation may be required.
Frequently Asked Questions
What is a learning curve?
A learning curve tracks model performance across growing training sizes. It compares training and validation behaviour. This helps identify bias, variance, and data needs.
Why can training performance decrease?
Small datasets are easier to memorise. Larger datasets contain more varied examples. Training scores can therefore fall while generalisation improves.
What does a large gap mean?
A large gap often indicates variance or overfitting. The model fits training data better than validation data. More data or stronger regularisation may help.
Can this calculator prove overfitting?
No single curve proves a diagnosis. The calculator provides indicators based on supplied results. Confirm findings with validation design and domain knowledge.
How are repeated scores entered?
Place several scores in one cell. Separate values with commas or semicolons. The calculator computes means, variation, and confidence intervals.
Which fitting method should I choose?
Logarithmic and power models often capture diminishing returns. Linear fits are easier to interpret. Compare fit quality before trusting long-range forecasts.
Can lower error metrics be used?
Yes, choose lower is better. Rankings and improvement directions will reverse automatically. Generalisation gaps still appear as absolute practical differences.
What is a plateau?
A plateau occurs when recent gains become very small. It can indicate limited benefit from more data. Model or feature changes may matter more.
Are future sample estimates exact?
No, they depend on the selected curve model. Extrapolation becomes uncertain beyond observed sizes. Treat sample forecasts as planning guidance only.