| Time | Model | Dataset | Dropout | Train | Validation | Test | Gap | Epochs | Score |
|---|---|---|---|---|---|---|---|---|---|
| No experiments yet. | |||||||||
Local leaderboard
| # | Player | Challenge | Score |
|---|
How dropout works
During training, dropout independently removes selected activations with probability p. Remaining activations are scaled so inference can use every neuron without changing the expected signal.
Generalisation gap
The gap compares training performance with validation performance. A large positive gap often signals memorisation, while weak scores on both sets can indicate underfitting.
Recommended starting ranges
Dense hidden layers often start around 0.10 to 0.50. Input dropout is usually lower. The best value depends on data size, noise, network capacity, and regularisation.
Other defences
L1 promotes sparse weights, L2 discourages large weights, and early stopping limits memorisation time. Excessive combined regularisation may underfit the model.
Formula and terminology
Mask: mᵢ ~ Bernoulli(1 − p). Training activation: h̃ᵢ = mᵢhᵢ/(1 − p). Expected active neurons: n(1 − p).
Co-adaptation: neurons depending too heavily on specific partners. Inverted dropout: scaling during training instead of inference.