DataTerminology.comNEW
A supervised learning model that recursively splits features into a tree structure to predict classes or continuous values.
DESCRIPTION
Decision trees segment feature space by recursive binary splits guided by impurity or error reduction. They support classification and regression, yielding interpretable rules. CART, ID3, and C4.5 are well-known algorithms differing in criteria and handling of features. Trees can overfit; pruning and depth limits mitigate this. Ensembles like Random Forests and Gradient Boosted Trees improve accuracy and robustness. Handling missing values, categorical encodings, and class imbalance are practical considerations. Trees are fast to train and predict, scaling to large datasets. Feature importance and path explanations aid interpretability. They remain a strong baseline in tabular ML.
ALTERNATIVE DEFINITIONS
scikit-learn
Decision Trees are non-parametric supervised learning methods used for classification and regression.
Emphasizes non-parametric nature and dual use for classification/regression.
Wikipedia
Decision tree learning uses a decision tree to go from observations about an item to conclusions about the item’s target value.
Frames decision trees as mapping observations to target values via a learned tree.
RELATED TERMS
Gradient Boosted Trees Classification and Regression Trees Gini Impurity Entropy Information Gain Pruning Explainability Model Overfitting Tree Depth
COMPETING TERMS
Random Forest Logistic Regression
RELATIONS
KNOWLEDGE AREAS
TARGET AUDIENCE
Data Scientist AI Engineer Data Analyst MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Decision tree learning
- XGBoost - Boosted tree methods reference
- LightGBM - Gradient boosting trees library docs
- Microsoft - Tree-based models in AutoML context
TOP QUESTIONS
When should I use a Decision Tree?
Use trees for tabular data where interpretability, nonlinearity, and handling of mixed feature types are needed. They are strong baselines.
Category: Opinion
How does pruning help Decision Trees?
Pruning reduces overfitting by removing splits with limited predictive value, improving generalization.
Category: Example
What split criteria are common in Decision Trees?
Gini impurity and entropy for classification; mean squared error for regression are typical choices.
Category: Definition
How do Decision Trees handle missing values?
Approaches include surrogate splits, imputation, or treating missing as a separate category, depending on implementation.
Category: Requirements
Why use ensembles instead of a single Decision Tree?
Ensembles like Random Forests and Gradient Boosted Trees reduce variance or bias, typically improving accuracy and robustness.
Category: Comparison
What is a common mistake with Decision Trees?
Allowing unbounded depth leads to overfitting. Set depth, min samples, or use pruning.
Category: Mistakes
Are Decision Trees interpretable?
Yes. Paths correspond to explicit rules, and feature importance summarizes influence, though deep trees can be cumbersome.
Category: Misunderstanding
Do Decision Trees require feature scaling?
No. Trees are invariant to monotonic transformations and don’t require normalization.
Category: Example
How do categorical features work with Decision Trees?
Some implementations natively support categorical splits; others require encoding like one-hot or target encoding.
Category: Requirements