DataTerminology.comNEW
Also known as: Poor Generalization, High Variance
When a model fits training noise so closely that performance degrades on unseen data.
DESCRIPTION
Overfitting occurs when a model is overly complex relative to the available data, capturing noise rather than underlying patterns. Symptoms include low training error but high validation/test error. Common causes are high-capacity models, insufficient data, data leakage, or poor regularization. Remedies include regularization (L1/L2), early stopping, data augmentation, dropout, pruning, and cross-validation. Proper validation and monitoring are essential to detect and prevent overfitting.
ALTERNATIVE DEFINITIONS
Wikipedia
In statistics and machine learning, overfitting is a modeling error which occurs when a function is too closely fit to a limited set of data points.
Frames overfitting as a modeling error due to closely fitting limited data.
Google Developers
Overfitting is when a model fits the training data too well but fails to generalize to new examples.
Highlights failure to generalize as defining characteristic.
IBM
Overfitting occurs when a statistical model describes random error or noise instead of the underlying relationship.
Emphasizes modeling noise rather than signal.
RELATED TERMS
Bias-Variance Tradeoff Cross-Validation Regularization Early Stopping Data Augmentation Feature Reduction Dropout Pruning Model Complexity Superclass
COMPETING TERMS
RELATIONS
- Broader: Overfitting
KNOWLEDGE AREAS
AI Data Science Mathematics Testing & Validation
TARGET AUDIENCE
Data Scientist AI Engineer MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Overfitting
- Google Developers - Crash Course section on overfitting and regularization
- scikit-learn - Cross-validation to detect and prevent overfitting
- Goodfellow, Bengio, Courville - Deep learning text discussing capacity and overfitting
TOP QUESTIONS
How do you detect Model Overfitting?
Look for a large gap between training and validation performance. Use learning curves and cross-validation to confirm high variance.
Category: Testing
What causes Model Overfitting most often?
Excessive model capacity relative to data volume or noise, coupled with weak regularization and data leakage, commonly causes overfitting.
Category: Overview
How can you prevent Model Overfitting?
Use regularization, early stopping, dropout, data augmentation, and simpler models. Collect more diverse data when possible.
Category: Requirements
Is Model Overfitting always bad?
Yes for generalization, but slight overfitting can be acceptable if validation performance meets requirements. Always monitor real-world performance.
Category: Opinion
How does cross-validation help with Model Overfitting?
It provides a more reliable estimate of generalization and reduces variance in performance assessment, exposing overfitting tendencies.
Category: Overview
What is the relationship between Overfitting and the bias-variance tradeoff?
Overfitting corresponds to high variance. Reducing variance often increases bias, so balance is key.
Category: Explanation
Can data augmentation reduce Overfitting?
Yes, by increasing effective training diversity, especially in vision and audio tasks. It regularizes without changing model architecture.
Category: Best practices
What is a hot topic in Overfitting mitigation?
Techniques like mixup, sharpness-aware minimization, and pretraining with foundation models help improve generalization.
Category: Topic
How does feature selection affect Overfitting?
Removing redundant or noisy features reduces model variance and risk of spurious patterns, improving generalization.
Category: Management