Formula Used
Regression loss: J = (1/n) Σ(ŷ − y)²
Logistic loss: J = −(1/n) Σ[y log(p) + (1 − y) log(1 − p)]
Gradient descent repeatedly moves parameters opposite the calculated loss gradient. The learning rate controls every update’s size and stability. Smaller steps often converge reliably but require more iterations.
Adaptive optimizers rescale updates using recent gradient information. Momentum reduces oscillation across narrow or uneven loss surfaces. Adam combines momentum with adaptive second-moment scaling for efficiency.
How to Use
- Select an objective function and optimisation method.
- Enter dataset rows with the target last.
- Choose learning, stopping, scaling, and stability settings.
- Enable gradient checking or optimizer comparison when needed.
- Submit the form and inspect convergence charts.
- Export the history for reporting or further analysis.
Example Data
| Feature | Target | Expected relationship |
|---|---|---|
| 1 | 3 | y = 2x + 1 |
| 2 | 5 | y = 2x + 1 |
| 3 | 7 | y = 2x + 1 |
| 4 | 9 | y = 2x + 1 |
Frequently Asked Questions
What does the learning rate control?
It controls how far parameters move during each update. Large values can speed training but may cause divergence. Small values improve stability while extending total training time.
Which optimizer should beginners choose?
Batch descent clearly demonstrates the basic optimisation process. Adam often works well across varied practical datasets. Comparing both reveals differences in speed and stability.
Why should features be scaled?
Scaling prevents large features from dominating gradient calculations. It also creates smoother updates across different parameter dimensions. Standardisation usually improves convergence for uneven feature ranges.
What indicates successful convergence?
The loss should decrease and eventually change very little. Gradient magnitude should also approach the selected tolerance. Stable parameter values provide another useful convergence signal.
What causes gradient descent divergence?
An excessive learning rate commonly makes updates overshoot minima. Unscaled features can also create unstable gradient magnitudes. Gradient clipping and decay schedules may restore stability.
How does mini-batch descent work?
It updates parameters using small groups of training rows. This balances noisy stochastic updates and expensive full batches. Batch size changes memory use and optimisation smoothness.
What does gradient checking verify?
It compares analytical gradients with finite-difference approximations. Small relative error suggests the implemented gradient is correct. Larger differences indicate formulas or data need review.
When should regularisation be enabled?
Regularisation helps reduce overly large or unstable model weights. L1 encourages sparsity while L2 smoothly shrinks parameters. Elastic Net combines both penalties using one ratio.
Can the calculator handle classification?
Yes, logistic mode accepts binary targets zero and one. It optimises cross-entropy loss and reports classification accuracy. Predicted probabilities use a threshold of one-half.