DataTerminology.comNEW
Linear regression with an L1 penalty that shrinks coefficients and sets some exactly to zero.
DESCRIPTION
Lasso adds an L1 norm penalty to linear regression, encouraging sparsity by driving some coefficients to zero. This yields models that perform variable selection and can improve generalization. It is useful when many features are irrelevant or correlated. Solutions are commonly computed via coordinate descent or proximal methods. Hyperparameter tuning controls regularization strength and the sparsity-accuracy tradeoff. Elastic Net blends L1 and L2 to handle collinearity and grouped features more robustly.
ALTERNATIVE DEFINITIONS
Wikipedia
In statistics and machine learning, lasso is a regression analysis method that performs both variable selection and regularization in order to enhance the prediction accuracy and interpretability of the statistical model it produces.
Highlights simultaneous variable selection and regularization benefits.
scikit-learn
The Lasso is a linear model that estimates sparse coefficients.
Emphasizes sparsity of coefficients directly.
Robert Tibshirani
We propose a new method for estimation in linear models. The lasso minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant.
Original formulation describing the L1-constrained optimization.
RELATED TERMS
Regularization Linear Model Feature Reduction Sparse Modeling Coordinate Descent Cross-Validation
COMPETING TERMS
KNOWLEDGE AREAS
TARGET AUDIENCE
Data Scientist AI Engineer MLOps Engineer Data Analyst
EXTERNAL SOURCES
- Wikipedia - Lasso (statistics)
- scikit-learn - API and theory for Lasso
- The Elements of Statistical Learning - Reference text covering L1 regularization
TOP QUESTIONS
When should I prefer Lasso over Ridge Regression?
Choose Lasso when you expect a sparse set of influential features; it performs embedded feature selection.
Category: Opinion
How do I tune Lasso effectively?
Use cross-validation over the regularization path and standardize features to balance sparsity and predictive performance.
Category: Best practices
What is the biggest misunderstanding about Lasso?
That it always identifies the true set of features; correlation and sample size can affect selection stability.
Category: Misunderstanding
What optimization methods solve Lasso?
Coordinate descent, proximal gradient, and LARS are common efficient approaches.
Category: Concepts
How does Elastic Net relate to Lasso?
Elastic Net combines L1 and L2 penalties, mitigating Lasso’s instability with correlated predictors.
Category: Concepts
What are common mistakes using Lasso?
Not scaling features, using a single split instead of cross-validation, and ignoring coefficient shrinkage bias.
Category: Mistakes
How do I interpret Lasso coefficients?
Non-zero coefficients indicate selected features, but magnitudes are biased toward zero due to regularization.
Category: Interpretation
Is Lasso suitable for high-dimensional data?
Yes, it is widely used when p >> n, but stability selection may be needed.
Category: Relevancy
Where should I start learning about Lasso?
Read scikit-learn’s linear models guide, then consult ESL’s chapters on regularization.
Category: Getting Started