DataTerminology.comNEW
Also known as: CART
A decision tree methodology that builds predictive models for classification and regression by recursively partitioning data to minimize impurity or error.
DESCRIPTION
CART, introduced by Breiman, Friedman, Olshen, and Stone, induces binary trees using impurity measures (e.g., Gini) for classification and variance reduction for regression. It supports cost-complexity pruning to balance fit and complexity. CART’s binary splits enable fast evaluation and are the basis of many ensemble methods like Random Forests and Gradient Boosted Trees. Its interpretability and strong baselines make it a staple in ML practice and education.
ALTERNATIVE DEFINITIONS
Wikipedia
Classification and regression trees (CART) are a type of decision tree learning technique that can be used for classification or regression predictive modeling problems.
Summarizes CART as a general decision tree approach for two task types.
scikit-learn
The classes DecisionTreeClassifier and DecisionTreeRegressor implement a greedy algorithm for learning classification and regression trees.
Connects CART-style learning to common implementations in practice.
RELATED TERMS
Decision Tree Gini Impurity Variance Reduction Pruning Random Forest Gradient Boosted Trees Model Overfitting Feature Importance ID3
COMPETING TERMS
RELATIONS
- Broader: Decision Tree
KNOWLEDGE AREAS
AI Data Science Testing & Validation
TARGET AUDIENCE
Data Scientist AI Engineer MLOps Engineer Data Analyst
EXTERNAL SOURCES
- Wikipedia - Decision tree learning
- scikit-learn - Decision tree API and theory overview.
- CRAN - rpart package manual for CART.
TOP QUESTIONS
What impurity measures are used in Classification and Regression Trees?
Gini impurity or entropy for classification, and variance or mean squared error for regression tasks are common.
Category: Definitions
How does pruning work in Classification and Regression Trees?
Cost-complexity pruning removes branches that contribute little to predictive power, balancing accuracy and simplicity.
Category: Mechanism
When should I use Classification and Regression Trees over linear models?
Use CART when relationships are nonlinear, include interactions, and interpretability of rules is desired. They handle mixed data types well.
Category: Opinion
What are common pitfalls in Classification and Regression Trees?
Overfitting without pruning, biased splits with high-cardinality features, and instability to small data perturbations are common issues.
Category: Mistakes
How does CART relate to Random Forests?
Random Forests are ensembles of CART models trained on bootstrapped samples with feature subsampling to reduce variance and improve accuracy.
Category: Relationships
Can Classification and Regression Trees handle missing values?
Yes, many implementations use surrogate splits or simple imputation strategies to handle missing features.
Category: Practice
What hyperparameters matter in Classification and Regression Trees?
Max depth, min samples per split/leaf, and impurity decrease thresholds control complexity and generalization.
Category: Practice
Are Classification and Regression Trees interpretable?
Yes, paths from root to leaves produce human-readable rules. However, very deep trees can become hard to interpret.
Category: Relevancy
How do I evaluate Classification and Regression Trees?
Use cross-validation and metrics suited to the task (accuracy, AUC, RMSE) and inspect learning curves and feature importances.
Category: Testing