DataTerminology.comNEW
Also known as: Adaptive Boosting
A boosting algorithm that iteratively reweights misclassified instances and combines weak learners to form a strong classifier.
DESCRIPTION
AdaBoost builds an ensemble by sequentially training weak learners, emphasizing previously misclassified examples. Each learner’s contribution is weighted based on its error. The final prediction aggregates learners, often improving margins and generalization. It commonly uses decision stumps as base learners. AdaBoost is sensitive to label noise and outliers. Variants and successors include Gradient Boosting and modern libraries like XGBoost.
ALTERNATIVE DEFINITIONS
Wikipedia
AdaBoost, short for Adaptive Boosting, is a machine learning ensemble meta-algorithm formulated by Yoav Freund and Robert Schapire.
Attributes originators and clarifies ensemble meta-algorithm nature.
scikit-learn
An AdaBoost classifier is a meta-estimator that begins by fitting a classifier on the original dataset and then fits additional copies of the classifier on the same dataset but where the weights of incorrectly classified instances are adjusted so that subsequent classifiers focus more on difficult cases.
Details the iterative reweighting mechanism in practical terms.
RELATED TERMS
Boosting XGBoost Weak Learner Decision Stump Ensemble Learning Weighting Model Overfitting Exponential Loss Margin
COMPETING TERMS
Gradient Boosting Random Forest
RELATIONS
- Broader: Boosting Ensemble Learning
KNOWLEDGE AREAS
TARGET AUDIENCE
Data Scientist AI Engineer MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - AdaBoost
- scikit-learn - AdaBoost algorithm docs
- XGBoost - Boosting concepts overview
TOP QUESTIONS
What base learners are common in AdaBoost?
Decision stumps (depth-1 trees) are popular due to simplicity and fast training.
Category: Practical
Why is AdaBoost sensitive to noise?
Mislabels receive increasing weights, causing the model to chase noisy observations and overfit.
Category: Risks
How do AdaBoost and Gradient Boosting differ?
AdaBoost reweights samples with exponential loss; gradient boosting optimizes differentiable loss via gradient steps.
Category: Comparison
What are key hyperparameters in AdaBoost?
Number of estimators and learning rate control ensemble size and contribution strength.
Category: Configuration
Can AdaBoost handle regression?
Yes, AdaBoost.R and variants adapt boosting to regression tasks using appropriate loss functions.
Category: Advanced
How to prevent AdaBoost overfitting?
Use early stopping via validation, tune learning rate, and limit number of estimators.
Category: Best practices
Is AdaBoost still relevant with modern methods?
Yes, it remains a strong baseline and educative stepping stone to advanced boosting libraries.
Category: Relevancy
What metrics best evaluate AdaBoost?
Use standard classification metrics; also inspect margins and calibration if probabilities matter.
Category: Evaluation
How does AdaBoost combine weak learners?
Learners are weighted by performance and aggregated, typically via a weighted vote or sum.
Category: Concept