Decision Tree Entropy Calculator

Calculation Result

Parent entropy

Best information gain

Best gain ratio

Parent purity

Rank	Candidate split	Weighted entropy	Information gain	Gain ratio	Gini after split	Status

Step-by-step calculation

Calculation settings

Input mode

Logarithm base

Decimal precision

Minimum child size

Useful gain threshold

Normalize parent probabilities automatically

Parent node classes

Add labels and values for every target class.

Candidate splits

Create binary or multiway splits and compare their quality.

CSV data tools

Use columns: split, child, then one column per class. The first row contains headings.

Paste CSV data

Upload CSV file

Examples and saved work

Example dataset

Formula used

H(S) = -Σ pᵢ log(pᵢ)

Weighted child entropy = Σ (|Sⱼ| / |S|) × H(Sⱼ)

Information gain = H(parent) - Weighted child entropy

Gain ratio = Information gain / Split information

How to use

Choose counts, probabilities, or percentages.
Enter every parent class label and value.
Add one or more candidate feature splits.
Add child groups and their class distributions.
Select logarithm base and output precision.
Calculate and compare information gain results.
Review charts, steps, warnings, and rankings.
Export or save the completed calculation.

Understanding the measurements

Entropy

Entropy measures uncertainty inside a classification node. A pure node has zero entropy. Balanced classes produce higher entropy.

Information gain

Information gain measures entropy reduction after splitting. Larger positive values indicate cleaner child groups. Zero gain means no impurity improvement.

Gain ratio

Gain ratio adjusts information gain for split complexity. It can reduce preference for many-valued features. Undefined ratios appear when split information equals zero.

Entropy versus Gini impurity

Both measurements evaluate class mixing within nodes. Entropy uses logarithms and supports information gain. Gini often requires slightly less computation.

Frequently asked questions

What does zero entropy mean?

Zero entropy means every observation belongs to one class. The node is completely pure. No classification uncertainty remains there.

What is the maximum entropy?

Maximum entropy occurs when classes have equal probabilities. Its value depends on class count. It also depends on logarithm base.

Is higher information gain better?

Higher information gain usually indicates a stronger split. Constraints can still reject small children. Validation rules remain important.

Why compare gain ratio?

Information gain can favor features with many groups. Gain ratio penalizes excessive fragmentation. This supports fairer candidate comparison.

Can the calculator handle three classes?

Yes, it supports any practical class count. Add classes using the provided button. Every child updates automatically.

Can a split have several children?

Yes, each candidate can contain multiple child groups. This supports categorical multiway splits. Binary splits remain fully supported.

How are zero probabilities handled?

The expression zero times log zero becomes zero. This follows the limiting definition. It prevents invalid numerical results.

Why must child totals match?

Children should partition the same parent observations. Mismatched totals indicate missing or duplicated records. The calculator displays a warning.

Can weighted observations be entered?

Yes, decimal values can represent observation weights. Use count mode for weighted totals. All formulas use those values directly.