DataTerminology.comNEW
Also known as: Lasso Regularization
A regularization technique that adds the sum of absolute coefficient values as a penalty to reduce overfitting and induce sparsity.
DESCRIPTION
L1 regularization constrains model complexity by penalizing the absolute magnitude of parameters. In linear models, it corresponds to Lasso, which can drive some coefficients exactly to zero, performing implicit feature selection. This sparsity property is valuable for interpretability and high-dimensional problems. L1 can be sensitive to correlated features, often selecting one among many. It is commonly used in regression and classification and combined with L2 in Elastic Net. Properly tuned via cross-validation, L1 reduces variance and improves generalization.
ALTERNATIVE DEFINITIONS
Wikipedia
In statistics and machine learning, lasso (least absolute shrinkage and selection operator) is a regression analysis method that performs both variable selection and regularization in order to enhance the prediction accuracy and interpretability of the resulting statistical model.
Highlights variable selection via sparsity as central to L1-based Lasso.
scikit-learn
The Lasso is a linear model that estimates sparse coefficients.
Emphasizes sparsity in linear models with L1 penalty.
RELATED TERMS
Regularization Lasso Elastic Net Feature Reduction Sparsity Model Overfitting Cross-Validation Logistic Regression Linear Model Penalty Term
COMPETING TERMS
KNOWLEDGE AREAS
Mathematics AI Data Science MLOps
TARGET AUDIENCE
Data Scientist AI Engineer MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Lasso (statistics)
- scikit-learn - Lasso model documentation
- Hastie, Tibshirani, Friedman - ESL chapters on regularization
- NeurIPS - Soft-thresholding and Lasso theory
TOP QUESTIONS
When should I use L1 Regularization?
Use L1 when you expect many irrelevant features and desire sparsity for interpretability. It helps in high-dimensional settings and can perform embedded feature selection.
Category: Application
How does L1 Regularization reduce overfitting?
By penalizing absolute coefficient magnitudes, L1 shrinks weights and can set some to zero, reducing model complexity and variance to improve generalization.
Category: Mechanism
What is a common pitfall of L1 Regularization?
With highly correlated predictors, L1 may arbitrarily select one feature and drop others. Elastic Net can mitigate this by combining L1 and L2 penalties.
Category: Mistakes
How do I tune L1 Regularization strength?
Use cross-validation to choose the regularization parameter (alpha or lambda) that balances bias and variance for best validation performance.
Category: Requirements
What is the main difference between L1 and L2 Regularization?
L1 encourages sparsity by zeroing coefficients; L2 spreads shrinkage across parameters without setting most exactly to zero.
Category: Comparison
Does L1 Regularization work with non-linear models?
Yes. L1 penalties are used with logistic regression, linear SVMs, and even neural networks, though optimization can be trickier due to non-differentiability at zero.
Category: Scope
What is the risk of over-regularizing with L1?
Too strong a penalty removes important features, causing high bias and poor predictive performance. Always validate on held-out data.
Category: Risks
Is L1 Regularization convex?
Yes, L1-penalized objectives are convex for linear models, enabling efficient optimization via coordinate descent or proximal methods.
Category: Theory
How does L1 Regularization affect interpretability?
It improves interpretability by reducing models to a small set of active features, making it clear which variables drive predictions.
Category: Benefits