DataTerminology.comNEW
An ensemble method that trains a meta-learner on out-of-fold predictions from base models to improve generalization.
DESCRIPTION
Stacked generalization builds multiple base models and uses their predictions as features for a meta-model. Out-of-fold training avoids leakage and yields robust meta-features. Stacking often improves accuracy by capturing complementary strengths across algorithms. It requires careful validation and feature pipelines to prevent overfitting. Practical implementations exist in scikit-learn and AutoML frameworks. Compared to bagging and boosting, stacking focuses on model combination via learned meta-weights.
ALTERNATIVE DEFINITIONS
Wikipedia
Stacked generalization is a method of using a high-level model to combine the outputs of several lower-level models trained on a dataset.
Defines the hierarchical combination of base model outputs via a meta-model.
scikit-learn
Stacking consists in training a meta-estimator to combine several base estimators.
Highlights training a meta-estimator to integrate base estimator predictions.
David H. Wolpert
Stacked generalization is a way of combining multiple generalizers to minimize their biases.
Original formulation emphasizing bias reduction via combination of learners.
RELATED TERMS
Ensemble Learning Meta-Learner Cross-Validation Blending Model Averaging Hyperparameter Tuning K-Fold Validation Generalization Error
COMPETING TERMS
RELATIONS
- Narrower: Blending
KNOWLEDGE AREAS
TARGET AUDIENCE
Data Scientist AI Engineer MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Stacked generalization
- scikit-learn - Stacking classifiers and regressors guide
- mlr3 - Stacking in R ML ecosystem
- Springer - Original stacked generalization paper
TOP QUESTIONS
Why use Stacked Generalization?
It often yields better accuracy by learning how to combine diverse base models’ strengths.
Category: Relevancy
What is a common mistake in Stacked Generalization?
Training the meta-model on predictions from the same folds causes leakage; use out-of-fold predictions.
Category: Mistakes
How does Stacked Generalization differ from Boosting?
Boosting reweights examples iteratively; stacking learns a meta-model to combine base predictions.
Category: Example
What risks does Stacked Generalization introduce?
Complexity and overfitting if base models are too similar or poorly validated.
Category: Risks
What is a competing term for Stacked Generalization?
Bagging aggregates models via averaging with bootstrap samples, focusing on variance reduction.
Category: Alternatives
How to implement Stacked Generalization properly?
Use k-fold cross-validation to generate out-of-fold meta-features and a separate holdout for final evaluation.
Category: HowTo
Where to start learning Stacked Generalization?
Read Wolpert’s paper, then explore scikit-learn’s Stacking classes with practical datasets.
Category: Requirements
What is a striking analogy for Stacked Generalization?
Like a panel of experts whose opinions are combined by a chairperson to reach a better decision.
Category: Example
What is the biggest misunderstanding about Stacked Generalization?
That any combination helps; diversity and correct validation are crucial for gains.
Category: Misunderstanding