DataTerminology.comNEW
Also known as: Gini Index (Decision Trees)
A decision tree splitting metric measuring the probability of misclassification when labeling according to class distribution in a node.
DESCRIPTION
Gini impurity quantifies how mixed the classes are in a node; lower values indicate purer nodes. CART commonly uses it to choose splits that maximize impurity decrease. It is computationally efficient and often performs comparably to entropy-based information gain. In ensembles like random forests, average impurity decreases across trees support feature importance measures. As with other greedy criteria, it may overfit without regularization. Class imbalance can also affect split behavior.
ALTERNATIVE DEFINITIONS
Wikipedia
Gini impurity measures how often a randomly chosen element from the set would be incorrectly labeled if it was randomly labeled according to the distribution of labels in the subset.
Classic definition focusing on misclassification probability based on class distribution.
scikit-learn
In CART, the Gini impurity is used to measure the quality of a split by minimizing the probability of misclassification.
Links Gini specifically to CART and split quality minimization.
RELATED TERMS
Decision Tree Classification and Regression Trees Random Forest Split Criterion Impurity Decrease Gain Ratio Pruning Feature Importance
COMPETING TERMS
KNOWLEDGE AREAS
TARGET AUDIENCE
Data Scientist AI Engineer MLOps Engineer Data Analyst
EXTERNAL SOURCES
- Wikipedia - Decision tree learning
- scikit-learn - Mathematical details of criteria
- The Elements of Statistical Learning - Tree-based methods overview
TOP QUESTIONS
When should I use Gini Impurity in trees?
Use it with CART for efficient, effective splits. It’s a strong default with performance comparable to information gain in many settings.
Category: Relevancy
How is Gini Impurity interpreted?
It’s the expected misclassification rate if classes are assigned according to their node frequencies. Lower is purer.
Category: Misunderstanding
Does Gini Impurity bias toward multi-valued features?
Like information gain, it can be influenced by cardinality. Use regularization or alternative criteria if this bias is problematic.
Category: Risks
How does Gini compare to Entropy?
Both measure impurity; Gini is simpler and often yields similar splits. Entropy has a stronger theoretical information-theoretic basis.
Category: Opinion
What affects Gini-based feature importance?
Correlated features and class imbalance can skew importance. Use permutation importance for validation.
Category: Mistakes
Is Gini Impurity suitable for multi-class problems?
Yes. It naturally extends by summing class probability products across all classes.
Category: Requirements
How do I prevent overfitting with Gini splits?
Limit tree depth, set min samples per split/leaf, and prune if supported. Cross-validate hyperparameters.
Category: Testing
Can Gini Impurity be used for regression?
No. Regression trees use criteria like variance reduction instead of classification impurity measures.
Category: Misunderstanding
How do random forests use Gini Impurity?
They average impurity decreases across many trees to assess split quality and derive feature importance heuristics.
Category: Example