DataTerminology.comNEW
An ensemble method that trains a meta-model to combine predictions from multiple base models for improved performance.
DESCRIPTION
Stacking, or stacked generalization, uses base learners to produce predictions that feed a meta-learner. To avoid leakage, meta-training uses out-of-fold predictions. The meta-learner learns how to weigh and correct base models’ strengths. Stacking often outperforms individual models when they are diverse and complementary. It can be applied to classification and regression. Careful cross-validation design is critical for robust performance.
ALTERNATIVE DEFINITIONS
Wikipedia
Stacked generalization is an ensemble-learning technique that uses a meta-model to combine the predictions of several base models.
Defines stacking by its meta-model combining mechanism.
scikit-learn
Stacking involves training a final estimator on the predictions of several base estimators.
API-level characterization focusing on estimators.
RELATED TERMS
Ensemble Learning Meta-Learner Out-of-Fold Predictions Cross-Validation Model Averaging Leakage Prevention Super Learner
COMPETING TERMS
KNOWLEDGE AREAS
TARGET AUDIENCE
Data Scientist AI Engineer MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Stacking (machine learning)
- scikit-learn - StackingClassifier/Regressor docs
- The Elements of Statistical Learning (online) - Ensemble learning theory
TOP QUESTIONS
How does Stacking reduce error?
It leverages diverse models to correct each other’s errors, with a meta-learner learning optimal combination rules.
Category: Concept
What prevents leakage in Stacking?
Use out-of-fold predictions for meta-training so base models never score on data they trained on.
Category: Best practices
How is Stacking different from Blending?
Blending typically uses a holdout for meta-training; stacking uses cross-validated out-of-fold predictions.
Category: Comparison
What base models work well in Stacking?
Diverse models (trees, linear, kernels, neural nets) with uncorrelated errors tend to combine best.
Category: Practical
What is a common mistake when applying Stacking?
Training the meta-learner on in-fold predictions, causing optimistic bias via leakage.
Category: Mistakes
What meta-model should I use in Stacking?
Start simple with linear or logistic models; escalate to nonlinear when base models are highly diverse.
Category: Guidance
Is Stacking interpretable?
Using linear meta-models and analyzing coefficients improves interpretability; complex meta-learners reduce transparency.
Category: Opinion
Can Stacking overfit?
Yes; mitigate with cross-validation, regularization, and limiting meta-model complexity.
Category: Risks
Where is Stacking most useful?
Competitions and high-stakes modeling where incremental performance gains justify complexity.
Category: Applications