DataTerminology.comNEW
Also known as: Bias Term, Intercept
Additive parameter in linear models that shifts the prediction independently of feature values, enabling affine rather than purely linear relationships.
DESCRIPTION
In regression and many neural models, the bias term (intercept) allows predictions when all features are zero by shifting the decision boundary or regression line. It improves model flexibility and often accelerates convergence. Centering features can reduce intercept magnitude, aiding numerical stability. Some models omit intercepts when data are pre-centered or constrained. Note: This entry concerns the intercept; different from statistical bias as systematic error.
ALTERNATIVE DEFINITIONS
Wikipedia
In statistics, particularly in regression analysis, an intercept is the expected mean value of the dependent variable when all independent variables are set to zero.
Defines intercept as the model’s baseline outcome at zero predictors.
scikit-learn
By default, LinearRegression will include an intercept in the model unless fit_intercept=False is specified.
Emphasizes default inclusion and configurability of intercept in tooling.
RELATED TERMS
Linear Model Logistic Regression Parameters Affine Transformation Feature Scaling Centering Regularization Mean Squared Error Model Overfitting Model Parameters
COMPETING TERMS
No-intercept Model
RELATIONS
- Broader: Parameters
- Narrower: Algorithmic Bias Bias (Mathematical) Group Attribution Bias Prediction Bias Sampling Bias Selection Bias
KNOWLEDGE AREAS
TARGET AUDIENCE
Data Scientist Data Analyst AI Engineer MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Intercept (statistics)
- scikit-learn - OLS with intercept parameterization
- Statsmodels - OLS example including intercept
- Penn State STAT 462 - Interpretation of intercept in regression
TOP QUESTIONS
Why include a Bias (intercept) in linear models?
It allows nonzero baseline predictions and improves fit when data are not centered. Without it, the model is forced through the origin, which can bias estimates.
Category: Relevancy
When should I use a no-intercept model instead of Bias?
Use no-intercept only when theory or preprocessing ensures the response is zero at zero predictors. Otherwise, include the intercept.
Category: Best practices
How does feature centering affect the Bias term?
Centering reduces intercept magnitude and can improve numerical stability, but the model’s predictive performance otherwise remains unchanged.
Category: Concepts
Does regularization penalize the Bias?
Most implementations do not penalize the intercept by default. Penalizing it can distort baseline estimates.
Category: Techniques
What is a common misunderstanding about Bias in machine learning?
Confusing intercept with statistical bias (systematic error). They are distinct concepts with different implications.
Category: Misunderstanding
How is Bias represented in neural networks?
As an additive term per neuron or layer, shifting activation thresholds for more flexible decision boundaries.
Category: Concepts
How do I interpret the Bias in logistic regression?
It represents the log-odds of the positive class when all features are zero, i.e., the baseline propensity.
Category: Interpretation
What is a risk of omitting the Bias term?
Forcing the fit through the origin can inflate errors and misrepresent relationships, especially with uncentered data.
Category: Risk
How does Bias interact with dummy variables?
With one-hot encoding, the intercept captures the baseline category’s effect. Drop-one encoding avoids multicollinearity with the intercept.
Category: Best practices