Momentum Optimiser Calculator

Optimisation results

Ready

Updated parameter

—

Current velocity

—

Applied update

—

Total movement

—

Average velocity norm

—

Maximum update norm

—

Direction changes

—

Convergence status

—

Gradient contribution

—

Momentum contribution

—

Average update norm

—

Maximum velocity norm

—

Average gradient norm

—

Parameter change

—

Displacement norm

—

Stabilisation step

—

Iteration details

Step	Learning rate	Momentum	Gradient	Previous velocity	Current velocity	Update	Previous parameter	New parameter	Update norm

Optimiser comparison

Method	Final parameter	Total movement	Maximum update	Direction changes

Parameter path

Velocity norm

Gradient norm

Update magnitude

Optimiser comparison path

Two-dimensional parameter trajectory

The trajectory illustrates parameter movement, not the actual loss surface.

Core optimiser settings

Parameter label

Initial parameter vector

Use commas for vectors, such as 1.2, -0.5.

Initial velocity vector

Momentum mode

Initial learning rate

Momentum coefficient

Optimisation steps

Decimal precision

Convergence tolerance

Gradient generator and input

Gradient source

Base gradient vector

Random seed

Custom gradient sequence

Use one step per line. Separate vector components with commas.

Example preset

Import gradient CSV

Learning-rate schedule

Schedule type

Decay rate

Step interval

Decay start step

Minimum learning rate

Learning-rate warm-up steps

Custom learning-rate sequence

Momentum, regularisation, and clipping

Dampening factor

Momentum warm-up steps

Starting momentum

Momentum decay

Custom momentum sequence

Weight decay

Regularisation mode

Gradient clipping

Clipping threshold

Gradient scale

Batch size

Average gradients by batch size

Maximise objective

Comparison settings

Momentum coefficients to compare

Learning rates to compare

Standard gradient descent

Selected momentum mode

Nesterov momentum

Formula used

Classical momentum:
vₜ = βvₜ₋₁ + (1 − d)gₜ
θₜ = θₜ₋₁ − ηvₜ

Symbols may follow different framework conventions. Review each selected formula carefully.

How to use

Enter scalar parameters or comma-separated vectors.
Select a gradient source and optimiser mode.
Configure schedules, clipping, and regularisation.
Run the calculation and review every update.
Export, print, copy, or save your results.

Worked example data

Input	Example	Purpose
Initial parameter	1.000000	Starting model weight
Gradient sequence	0.50, 0.40, 0.25	Loss derivatives by step
Learning rate	0.010000	Controls each update size
Momentum	0.900000	Retains previous update direction
Initial velocity	0.000000	Starts the momentum buffer

Momentum optimiser guidance

Why momentum helps

Momentum accumulates gradients moving consistently. It can accelerate progress along shallow directions.

Oscillation control

Repeated opposing gradients reduce stored velocity. This can stabilise movement across narrow valleys.

Overshooting risk

Large learning rates and momentum amplify updates. Reduce either value when divergence appears.

Nesterov behaviour

Nesterov applies a look-ahead correction. Supplied gradients should represent that anticipated position.

Frequently asked questions

What does the momentum coefficient control?

It controls how strongly previous velocity influences new updates.

Why is 0.9 commonly used?

It often balances acceleration and stability across many training tasks.

Can momentum exceed one?

Values near or above one can create unstable growing updates.

What is the velocity buffer?

It stores accumulated gradient direction from earlier optimisation steps.

How does dampening change momentum?

Dampening reduces each new gradient contribution entering the buffer.

When should gradients be clipped?

Clip gradients when occasional large values create unstable updates.

What does decoupled weight decay do?

It shrinks parameters separately from the optimiser gradient calculation.

Why compare several learning rates?

The comparison reveals sensitivity, overshooting, and slow convergence risks.

Does this replace framework training?

No. It provides educational simulation and update verification only.

Can vectors have several parameters?

Yes. Every vector must use matching component dimensions.