DataTerminology.comNEW
A modeling error where a model captures noise in training data, harming performance on unseen data.
DESCRIPTION
Overfitting arises when a model memorizes idiosyncrasies of the training set rather than learning generalizable patterns. It manifests as low training error and high validation/test error. Common mitigations include regularization, cross-validation, early stopping, data augmentation, and simplifying the model. Monitoring generalization via validation curves and robust evaluation prevents over-optimistic assessments. Balanced model capacity and sufficient, representative data reduce risk.
ALTERNATIVE DEFINITIONS
Google Machine Learning Crash Course
Overfitting occurs when your model captures the noise in the training data to the extent that it negatively impacts the model’s performance on new data.
Highlights harm to generalization from learning noise.
scikit-learn
Overfitting happens when the model performs well on the training data but poorly on unseen data due to modeling the noise in the training samples.
Contrasts training and test performance to diagnose overfitting.
RELATED TERMS
Regularization Cross-Validation Bias-Variance Tradeoff Early Stopping Dropout Data Augmentation Superclass Model Complexity Validation Set
COMPETING TERMS
RELATIONS
- Narrower: Model Overfitting
KNOWLEDGE AREAS
AI Data Science Testing & Validation MLOps
TARGET AUDIENCE
Data Scientist AI Engineer MLOps Engineer Data Engineer
EXTERNAL SOURCES
- Wikipedia - Overfitting
- scikit-learn - Visualization of under/overfitting
- Google Developers - MLCC module on overfitting
TOP QUESTIONS
What causes Overfitting?
Excessive model complexity, insufficient or unrepresentative data, leakage, and inadequate regularization commonly cause overfitting.
Category: Requirements
How do I detect Overfitting early?
Track training vs. validation curves. Diverging performance indicates overfitting; apply early stopping or stronger regularization.
Category: Usage
What techniques reduce Overfitting?
Use regularization (L1/L2), dropout, data augmentation, cross-validation, and simpler architectures.
Category: Alternatives
Is Overfitting always bad?
Yes for generalization; however, specialized tasks with fixed distributions may tolerate more fitting if deployment distribution matches training.
Category: Opinion
How does data augmentation help Overfitting?
It increases effective data diversity, discouraging memorization and improving generalization.
Category: Topic
Can regularization be too strong?
Yes. Excessive regularization can cause underfitting, reducing both training and test performance.
Category: Risks
How does cross-validation address Overfitting?
It provides robust out-of-sample estimates, aiding hyperparameter selection and model comparison with less bias.
Category: Usage
What is data leakage’s role in Overfitting?
Leakage inflates training/validation performance by exposing future or target information, masking overfitting until production.
Category: Risks
How does Overfitting relate to the bias-variance tradeoff?
Overfitting corresponds to high variance; techniques that reduce variance (e.g., regularization, ensembling) help restore balance.
Category: Topic