DataTerminology.comNEW
Also known as: LAD Regression
A regression method minimizing the sum of absolute residuals, yielding robustness to outliers.
DESCRIPTION
Least Absolute Deviations (LAD) regression minimizes the L1 norm of residuals rather than the L2 norm. This makes LAD more robust to outliers than ordinary least squares and aligns with median-based estimators. LAD connects closely to quantile regression, especially at the median. Optimization often uses linear programming or specialized algorithms like iteratively reweighted least squares. LAD is useful when error distributions are heavy-tailed or measurement noise is non-Gaussian.
ALTERNATIVE DEFINITIONS
Wikipedia
Least absolute deviations (LAD), also referred to as L1-norm regression, is a statistical method for estimating parameters by minimizing the sum of absolute differences between observed and predicted values.
Highlights the L1 norm objective and robustness properties.
Wolfram MathWorld
Least absolute deviations is a regression method that minimizes the sum of absolute residuals, yielding parameter estimates that are robust to outliers.
Emphasizes robustness to outliers in regression estimation.
NIST Engineering Statistics Handbook
Least Absolute Deviations (LAD) regression fits a model by minimizing the sum of absolute deviations of the residuals, providing resistance to the influence of extreme values.
Focuses on resistance to extreme values within a practical engineering handbook context.
RELATED TERMS
L1 Loss Median Regression Quantile Regression Robust Regression Huber Loss Outliers Linear Model Optimization Lasso
COMPETING TERMS
Least Squares Huber Regression
KNOWLEDGE AREAS
TARGET AUDIENCE
Data Scientist AI Engineer Data Analyst
EXTERNAL SOURCES
- Wikipedia - Least absolute deviations
- NIST - Engineering handbook entry on LAD
- Wolfram MathWorld - Mathematical overview of LAD
- Statsmodels - Notebook linking LAD to quantile regression
TOP QUESTIONS
When should Least Absolute Deviations be preferred over Least Squares?
Choose LAD when data contain outliers or heavy-tailed errors. LAD’s L1 loss reduces the influence of extreme residuals compared to the L2 loss in least squares.
Category: Requirements
How do you compute Least Absolute Deviations efficiently?
Use linear programming solvers, coordinate descent, or iteratively reweighted least squares. For median regression, specialized algorithms are efficient.
Category: Management
What is a common misunderstanding about LAD?
People assume LAD always outperforms least squares; in low-noise Gaussian settings, least squares is optimal and usually better.
Category: Misunderstanding
How does LAD connect to quantile regression?
Median (50th percentile) quantile regression corresponds to LAD, minimizing absolute residuals. Other quantiles adjust asymmetric penalties.
Category: Example
What are typical risks when applying LAD?
Optimization can be slower and less numerically smooth. If errors are truly Gaussian, LAD may be less efficient than least squares.
Category: Risks
Which metrics evaluate LAD regression?
Use MAE, median absolute error, or quantile loss. Residual plots help assess robustness and model fit.
Category: Validation
What features affect LAD performance most?
Proper feature scaling and robust feature engineering matter. High-leverage points still need attention despite LAD’s robustness.
Category: Mistakes
Is LAD suitable for high-dimensional data?
Yes, but consider L1-regularization (Lasso) for feature selection and stability. Optimization complexity increases with dimension.
Category: Opinion
What is a competing method to Least Absolute Deviations?
Huber regression blends L1 and L2 losses for balanced robustness and efficiency, while least squares is optimal under Gaussian noise.
Category: Alternatives