Formula Used
Entropy: H(S) = −Σ pᵢ logb(pᵢ)
Weighted child entropy: Hsplit = Σ (|Sⱼ| / |S|)H(Sⱼ)
Information gain: IG(S,A) = H(S) − Hsplit
Gain ratio: Gain Ratio = Information Gain / Split Information
Gini reduction: Gini(parent) − Σ weightⱼ × Gini(childⱼ)
How to Use
- Select manual counts, probabilities, or CSV dataset analysis.
- Enter class distributions and candidate child-node distributions.
- For CSV data, enter exact target and feature headers.
- Choose automatic or manual thresholds for numerical features.
- Submit the form and review entropy, gain, charts, and rankings.
Example Data
| Outlook | Temperature | Humidity | Wind | Play |
|---|---|---|---|---|
| Sunny | 85 | 85 | Weak | No |
| Sunny | 80 | 90 | Strong | No |
| Overcast | 83 | 86 | Weak | Yes |
| Rain | 70 | 96 | Weak | Yes |
| Rain | 68 | 80 | Weak | Yes |
Frequently Asked Questions
What is information gain?
Information gain measures entropy reduction after splitting data. Larger values indicate cleaner class separation. Decision trees often prefer features with stronger gain.
Can information gain be negative?
Correctly weighted splits should not produce meaningful negative gain. Tiny negative values may appear from rounding. Large negative values usually indicate inconsistent inputs.
What does zero information gain mean?
Zero gain means the split preserves existing uncertainty. The child distributions resemble the parent distribution. That feature provides no useful separation.
Why calculate gain ratio?
Information gain can favor features with many categories. Gain ratio divides gain by split information. This adjustment can reduce that preference.
How are numerical thresholds selected?
The calculator sorts numeric values and tests midpoint thresholds. Each threshold creates left and right nodes. The highest information gain becomes the recommended threshold.
How are missing values handled?
You can ignore rows, create a missing category, or impute values. Numeric imputation uses the mean. Categorical imputation uses the mode.
What logarithm base should I choose?
Base two expresses entropy in bits. Natural logarithms express entropy in nats. Rankings remain consistent when one base is applied throughout.
What is a pure node?
A pure node contains only one target class. Its entropy and Gini impurity equal zero. Pure nodes require no further separation.
Should I choose the highest information gain?
The highest gain is often a strong candidate. Also inspect sample sizes, gain ratio, and overfitting risk. Practical model validation should confirm the final split.