DataTerminology.comNEW
Also known as: Collinearity, Correlated Predictors
A condition where predictor variables are highly correlated, leading to unstable estimates and unreliable inference in regression models.
DESCRIPTION
Multicollinearity occurs when one predictor is linearly dependent on others, making coefficient estimates unstable and standard errors large. It undermines interpretability and significance tests without necessarily harming prediction accuracy. Detection methods include VIF, correlation analysis, eigenvalue condition numbers, and regularization paths. Remedies include feature selection, combining variables, regularization (ridge), or dimensionality reduction (PCA).
ALTERNATIVE DEFINITIONS
Wikipedia
In statistics, multicollinearity is a phenomenon in which one predictor variable in a multiple regression model can be linearly predicted from the others with a substantial degree of accuracy.
Highlights linear dependence among predictors within regression.
Penn State Eberly College of Science
Multicollinearity occurs when two or more predictors in a regression model are highly correlated, meaning that one can be linearly predicted from the others with a substantial degree of accuracy.
Clarifies regression context and linear predictability among predictors.
RELATED TERMS
Variance Inflation Factor Ridge Regression Lasso Principal Component Analysis Regularization Feature Reduction Correlation Condition Number Linear Model Model Overfitting Correlation Matrix Regression Analysis Dummy Variable Trap
COMPETING TERMS
Orthogonal Predictors Principal Component Regression Orthogonal Features
KNOWLEDGE AREAS
Mathematics Data Science Data Modeling
TARGET AUDIENCE
Data Scientist Data Analyst Data Modeler
EXTERNAL SOURCES
- Wikipedia - Multicollinearity
- UCLA OARC - Notes on multicollinearity
- Penn State - Regression diagnostics and collinearity
TOP QUESTIONS
What is Multicollinearity?
It is high correlation among predictors, causing unstable regression coefficients and unreliable inference.
Category: Definition
How do you detect Multicollinearity?
Use Variance Inflation Factors, correlation matrices, and condition indices to assess predictor redundancy.
Category: Testing
What are the consequences of Multicollinearity?
Coefficient estimates become unstable, standard errors inflate, and interpretability suffers.
Category: Risks
How can Multicollinearity be addressed?
Apply ridge or lasso regularization, remove redundant variables, or use PCA-based regression.
Category: Best practices
Is Multicollinearity always harmful?
For prediction, it may be tolerable; for interpretation, it often undermines clarity.
Category: Opinion
What is a VIF threshold?
Rules of thumb use VIF > 5 or 10, but context matters and requires judgment.
Category: Requirements
Does Multicollinearity affect model selection?
Yes. It can bias criteria and degrade stability; consider regularization or dimensionality reduction.
Category: Topic
How does scaling interact with Multicollinearity?
Scaling does not remove collinearity, but aids regularized methods and numerical stability.
Category: Misunderstanding
Where to learn more about Multicollinearity?
See UCLA and Penn State regression diagnostics resources for practical guidance.
Category: Overview