Performance heatmap results
Ranked combinations
Formula used
Mean score = Σsᵢ / n Sample standard deviation = √[Σ(sᵢ − s̄)² / (n − 1)] Relative improvement (%) = [(current − baseline) / |baseline|] × 100 Generalization gap = training score − validation score Confidence interval = mean ± critical value × standard error
How to use this calculator
- Paste long-format experiment results or upload a CSV file.
- Select two hyperparameters and one numeric performance metric.
- Choose aggregation, optimization, filters, and comparison settings.
- Adjust normalization and colors without changing raw scores.
- Generate the heatmap and inspect ranked parameter combinations.
- Export the chart, data, summary, or reusable settings.
Example data structure
| learning_rate | batch_size | validation_accuracy | fold | model |
|---|---|---|---|---|
| 0.001 | 32 | 0.918 | 1 | CNN |
| 0.001 | 32 | 0.912 | 2 | CNN |
| 0.01 | 64 | 0.861 | 1 | CNN |
Frequently asked questions
What does each heatmap cell represent?
Each cell represents one hyperparameter combination. Its value uses the selected aggregation. Its color reflects the configured display transformation.
Why aggregate repeated trials?
Repeated runs contain random variation. Aggregation estimates typical performance. Standard deviation shows stability across repeated observations.
Should accuracy always be maximized?
Accuracy usually increases with improvement. Loss and error metrics should decrease. Automatic mode checks common metric names.
Does normalization change exported raw scores?
No raw metric values are replaced. Normalization only controls heatmap coloring. Tooltips and tables retain aggregated raw values.
How are missing combinations handled?
Missing combinations receive a separate color. They remain excluded from statistics. The summary reports their total count.
What is a stable parameter region?
Stable regions contain several near-best combinations. They reduce dependence on one exact setting. Tolerance controls the accepted performance difference.
How does comparison mode work?
Select a grouping column and two values. The calculator compares matching parameter cells. It shows absolute or percentage differences.
Can categorical hyperparameters be used?
Yes, categorical values are supported. Input order can remain unchanged. Text sorting is also available.
What causes a logarithmic error?
Logarithms require positive values. Zero and negative scores are invalid. Choose another transformation for those metrics.
How is the confidence interval calculated?
The interval uses repeated cell observations. Critical values approximate normal confidence levels. Small samples should be interpreted carefully.
Why might the best cell be misleading?
One score can reflect sampling noise. Compare variability and nearby cells. Prefer robust regions that generalize consistently.