Results
Choose a mode and calculate entropy.
Calculator settings
Detailed data table
Sort or filter the current calculation output.
Formula used
H(X) = -Σ pᵢ log_b(pᵢ)Entropy measures uncertainty within a probability distribution. Pure nodes have zero entropy. Balanced classes produce larger entropy values.
Decision-tree information gain subtracts weighted child entropy from parent entropy. Better splits reduce uncertainty more strongly. Gain ratio can reduce preference for highly fragmented splits.
Information Gain = H(parent) − [nₗ/n · H(left) + nᵣ/n · H(right)]How to use
- Select binary, multiclass, dataset, split, or comparison mode.
- Enter probabilities, class counts, or labeled feature data.
- Choose the logarithm base and visualization settings.
- Calculate to view metrics, curves, markers, and detailed rows.
- Export the result, chart, table, configuration, or report.
Example data
| Feature value | Class label | Sample weight |
|---|---|---|
| 1.2 | No | 1 |
| 1.8 | No | 1 |
| 2.4 | Yes | 1 |
| 3.0 | Yes | 1.5 |
| 3.8 | Yes | 1 |
Frequently asked questions
What does entropy measure in machine learning?
Entropy measures uncertainty or class disorder. A pure class distribution has zero entropy.
Why is binary entropy highest at probability 0.5?
Both outcomes are equally likely there. The model therefore has maximum uncertainty.
Which logarithm base should I use?
Base two reports bits. Natural logarithms report nats. Comparisons remain valid when one base is used consistently.
What is normalized entropy?
Normalized entropy divides entropy by its maximum possible value. This produces a scale from zero through one.
How does information gain select a split?
It measures uncertainty reduction after splitting. The threshold with larger gain usually separates classes better.
What is gain ratio?
Gain ratio divides information gain by split information. It can discourage overly fragmented decision-tree splits.
How is entropy different from Gini impurity?
Both quantify impurity. Gini is computationally simpler, while entropy has an information-theoretic interpretation.
Can this calculator use sample weights?
Yes. Split mode accepts an optional third weight column for weighted counts and entropy.
How are zero probabilities handled?
The mathematical limit of zero times log zero is treated as zero. This prevents numerical errors.