Learning Rate Decay Calculator for Machine Learning

Calculated Results

Ready for calculation

Current learning rate

—

Final projected rate

—

Percentage reduction

—

Current multiplier

—

Average learning rate

—

Schedule minimum

—

Schedule maximum

—

Decay progress

—

Training position

—

Cycle and warm-up

—

Learning-rate chart

Training Configuration

Optimiser

Primary schedule

Schedule unit

Initial learning rate

Minimum learning rate

Maximum learning rate

Total epochs

Current epoch

Dataset size

Batch size

Gradient accumulation

Steps per epoch override

Use zero for automatic calculation.

Total update steps override

Current update step

Zero derives the value from current epoch.

Current schedule position

Decay Parameters

Decay rate

Decay factor

Decay interval

Polynomial power

Decay start

Decay end

Exponential mode

Apply staircase updates

Warm-up length

Warm-up method

Warm-up starting rate

Decay after warm-up

Apply warm-up to any schedule

Clamp to minimum learning rate

Resume offset

Freeze ranges

Cycles, Restarts, Plateaus, and Milestones

Initial cycle length

Cycle multiplier

Restart peak multiplier

One-cycle increasing phase (%)

Cyclical mode

Cyclical gamma

Plateau reduction factor

Plateau patience

Epochs without improvement

Cooldown period

Improvement threshold

Milestones

Milestone factors

Each factor multiplies the previous learning rate.

Custom expression

Allowed variables: lr0, min, max, rate, factor, t, T, epoch, step, progress, Math.

Output Options

Decimal precision

Number notation

Chart vertical scale

Table row mode

Table interval

Selected checkpoints

Chart sample points

Use one-based numbering

Schedule Table

Position	Epoch	Update step	Learning rate	Multiplier	Initial rate %	Warm-up	Cycle	Restart
Calculate a schedule to populate this table.

Formula Used

Select a schedule and calculate its values.

The calculator applies optional warm-up, offsets, minimum limits, freezes, milestones, and restart rules after evaluating the selected schedule.

How to Use

Choose an optimiser and learning-rate schedule.
Enter training size, epochs, steps, and learning-rate limits.
Configure decay, warm-up, restart, or milestone parameters.
Select comparison schedules and output preferences.
Calculate, inspect the chart, and export the schedule.

Example Training Scenarios

Scenario	Suggested schedule	Typical configuration	Why it helps
Image classification	Cosine annealing	0.1 to 0.0001 over 100 epochs	Provides smooth late-stage refinement.
Transformer training	Warm-up plus cosine	5,000 warm-up steps and long decay	Stabilises early optimisation.
Fine-tuning	Linear decay	Small initial rate and short schedule	Reduces destructive parameter updates.
SGD training	Step decay	Multiply by 0.1 at milestones	Matches common benchmark recipes.
Fast convergence	One-cycle	Rapid rise and long annealing phase	Explores higher rates safely.

Frequently Asked Questions

What is learning-rate decay?

It gradually reduces the optimiser learning rate during training. Smaller updates can improve stability near a minimum. The schedule controls when and how reduction occurs.

Should decay use epochs or steps?

Epoch schedules are easier to interpret. Step schedules offer finer control for large datasets. Both approaches work when configured consistently.

Why use warm-up?

Warm-up begins with a smaller learning rate. It limits unstable early parameter updates. Transformers often benefit from this approach.

What is cosine annealing?

Cosine annealing reduces rates along a smooth cosine curve. It spends more time near both endpoints. This can improve final convergence.

When are warm restarts useful?

Restarts periodically raise the learning rate again. They may help leave narrow local regions. Later cycles can use smaller peaks.

How low should the final rate be?

The best minimum depends on model and optimiser. Very small values can stop meaningful learning. Validation behaviour should guide the choice.

What does reduce on plateau do?

It lowers the rate after progress stalls. Patience prevents reacting to temporary noise. A minimum rate prevents endless reduction.

Can schedules be compared directly?

Yes, this calculator overlays selected schedules. Shared settings make the curves comparable. Review averages, extremes, and current values.

Does optimiser choice change the formula?

The optimiser selector provides planning context. It does not silently alter schedule mathematics. You retain control over every rate.

Learning Rate Decay Calculator