Calculated Results
Results update after calculation or preset loading.
Recommendations and diagnostics
Generated data preview
Formula used
Exponential decay: ηₜ = η₀e−kt
Step decay: ηₜ = η₀γ⌊t/s⌋
Inverse-time decay: ηₜ = η₀ / (1 + kt)
Polynomial decay: ηₜ = (η₀ − ηmin)(1 − t/T)p + ηmin
Cosine annealing: ηₜ = ηmin + 0.5(ηmax − ηmin)(1 + cos(πt/T))
Linear warmup: ηₜ = ηmax(t/Tw)
How to use
- Select schedule mode or range-test mode.
- Choose a method and enter valid parameters.
- Optionally import custom learning-rate or loss data.
- Enable comparison schedules and customize the chart.
- Press Calculate, then inspect metrics and recommendations.
- Export the data, image, JSON, CSV, or printable PDF.
Worked example
A warmup-cosine schedule starts at 0.00001 and rises toward 0.003 during five epochs. It then decays smoothly toward 0.00001 across the remaining training period. This can stabilize early updates while preserving useful progress later.
| Epoch | Learning rate | Interpretation |
|---|---|---|
| 0 | 0.00001 | Warmup begins |
| 5 | 0.00300 | Peak rate reached |
| 50 | About 0.00150 | Cosine decay continues |
| 100 | 0.00001 | Training finishes gently |
Learning-rate curve guidance
Why schedules matter
Large rates can speed early learning but may cause unstable updates. Smaller rates support fine adjustments near convergence. A schedule balances exploration, speed, and stability.
Learning rate versus loss
The learning-rate curve shows the optimizer step size over time. The loss curve shows prediction error during training. A range test combines both to identify a useful rate region.
Warmup, cycles, and restarts
Warmup reduces early instability, especially with large batches. Cycles periodically raise and lower the rate. Restarts can help the optimizer leave shallow regions.
Common mistakes
A rate can decay too early, remain too high, or become effectively zero. Warmup can also consume too much training. Compare curves before committing expensive training time.
Frequently asked questions
What does a learning-rate curve show?
It shows how the optimizer learning rate changes during training.
Is this the same as a learning curve?
No. A learning curve usually displays training or validation performance.
When should warmup be used?
Warmup is useful for large batches, transformers, and unstable early training.
Why use logarithmic scaling?
Learning rates often span several orders of magnitude.
What is a good Adam learning rate?
It depends on the model and data, so test a reasonable range.
What is the one-cycle policy?
It raises the rate toward one peak, then decreases it strongly.
How is the suggested range-test rate found?
The calculator evaluates smoothed loss gradients and minimum-loss regions.
Can I compare several schedules?
Yes. Enable comparison cards and select independent methods.
Can custom CSV data be used?
Yes. Import x, learning-rate, and optional loss columns.
Does the calculator train a real model?
No. It visualizes schedules and analyzes supplied or simulated values.