DataTerminology.comNEW
Also known as: Tikhonov Regularization
A linear regression method that adds an L2 penalty to shrink coefficients and reduce overfitting.
DESCRIPTION
Ridge regression augments ordinary least squares with an L2 penalty on coefficient magnitudes to control model complexity. It is effective against multicollinearity, improving stability and predictive performance. The penalty parameter tunes the strength of shrinkage, often selected via cross-validation. Ridge does not drive coefficients to zero, unlike Lasso, making it useful when many small effects exist. Bayesian interpretations link ridge to Gaussian priors on coefficients. It is widely available in libraries like scikit-learn and statsmodels.
ALTERNATIVE DEFINITIONS
scikit-learn
Ridge regression addresses some of the problems of Ordinary Least Squares by imposing a penalty on the size of coefficients.
Practical library perspective emphasizing coefficient shrinkage.
Wikipedia
Ridge regression is a method of estimating the coefficients of multiple-regression models in scenarios where independent variables are highly correlated.
Highlights use for multicollinearity in multiple regression.
Penn State STAT Online
Ridge regression adds a penalty term proportional to the sum of squared coefficients to the least squares objective.
Formal definition via objective function modification.
RELATED TERMS
Elastic Net Linear Model Regularization Bias-Variance Tradeoff Cross-Validation Multicollinearity Shrinkage Methods Bayesian Linear Regression
COMPETING TERMS
Lasso Regression Linear Regression
RELATIONS
- Broader: Regression
KNOWLEDGE AREAS
TARGET AUDIENCE
Data Scientist AI Engineer MLOps Engineer Data Analyst
EXTERNAL SOURCES
- Wikipedia - Ridge regression
- scikit-learn - Ridge estimator documentation
- The Elements of Statistical Learning - Textbook chapter on shrinkage methods
- Statsmodels - OLS baseline for comparison
TOP QUESTIONS
When should Ridge Regression be preferred over Lasso Regression?
Use ridge when many features have small, nonzero effects and multicollinearity is high. Lasso is better for sparsity and feature selection.
Category: Opinion
How do you choose the regularization strength in Ridge Regression?
Use cross-validation to select the alpha parameter that optimizes validation performance and balances bias-variance.
Category: BestPractice
What is the main risk of Ridge Regression?
Over-regularization can bias estimates and underfit data, reducing predictive accuracy.
Category: Risks
Does Ridge Regression perform feature selection?
No, coefficients are shrunk but typically remain nonzero. It controls magnitude without producing sparse solutions.
Category: Misunderstanding
How does Ridge Regression handle multicollinearity?
The L2 penalty stabilizes coefficient estimates by discouraging large values correlated predictors otherwise induce.
Category: Concept
Is Ridge Regression scale-sensitive?
Yes; standardize features so the penalty treats coefficients comparably across scales.
Category: Requirements
Can Ridge Regression be interpreted in a Bayesian way?
Yes; it corresponds to MAP estimation with Gaussian priors on coefficients.
Category: Advanced
How does Ridge affect the bias-variance tradeoff?
It increases bias but reduces variance, often improving generalization.
Category: Concept
What metrics are suitable for evaluating Ridge Regression?
Common choices include RMSE, MAE, and R-squared on validation data.
Category: Metrics