DataTerminology.comNEW
Also known as: BIC, Schwarz Information Criterion
A criterion comparing models by penalizing complexity while rewarding goodness of fit.
DESCRIPTION
The Bayesian Information Criterion (BIC) evaluates candidate models by combining likelihood with a penalty proportional to the number of parameters and sample size. Lower BIC values indicate preferred models under this trade-off. It is asymptotically consistent, often favoring simpler models as data grows. BIC is widely used for regression, time series, and clustering model selection. Compared to AIC, BIC penalizes complexity more strongly. It assumes the true model is among candidates and relies on large-sample approximations.
ALTERNATIVE DEFINITIONS
Wikipedia
In statistics, the Bayesian information criterion (BIC) is a criterion for model selection among a finite set of models; the model with the lowest BIC is preferred.
Highlights the finite set of models and selection by minimum BIC.
Penn State Eberly College of Science
The Bayesian Information Criterion (BIC) is a criterion for model selection that balances model fit and complexity; lower values indicate a better model according to this criterion.
Emphasizes the balance of fit and complexity and the lower-is-better rule.
RELATED TERMS
Model Selection Likelihood Maximum Likelihood Estimation Model Overfitting Cross-Validation Bayes Factor Minimum Description Length Regularization Deviance Information Criterion
COMPETING TERMS
KNOWLEDGE AREAS
TARGET AUDIENCE
Data Scientist AI Engineer MLOps Engineer Data Analyst
EXTERNAL SOURCES
- Wikipedia - Bayesian information criterion
- Penn State Online Stat - Educational notes on BIC for model selection
- Statsmodels - BIC computation in statsmodels
- CRAN mclust - BIC for Gaussian mixture selection
TOP QUESTIONS
When should I use Bayesian Information Criterion?
Use BIC to compare models with different parameter counts on the same dataset. It is useful when you seek parsimonious models and have moderate to large sample sizes.
Category: Requirements
How does Bayesian Information Criterion differ from AIC?
BIC imposes a stronger penalty for additional parameters than AIC, often selecting simpler models as sample size grows. AIC focuses more on predictive accuracy than consistency.
Category: Example
What is minimized when using Bayesian Information Criterion?
You minimize the BIC value across candidate models. The model with the lowest BIC is considered preferable under this criterion.
Category: Misunderstanding
Is Bayesian Information Criterion truly Bayesian?
Despite its name, BIC arises from approximations to the Bayes factor and uses maximum likelihood estimates, not full Bayesian inference with priors over parameters.
Category: Opinion
Can Bayesian Information Criterion be used for non-nested models?
Yes, BIC can compare both nested and non-nested models, as long as likelihoods are comparable on the same dataset.
Category: Requirements
What are common mistakes with Bayesian Information Criterion?
Comparing BIC across different datasets or ignoring model misspecification are common pitfalls. Ensure identical data and consider whether candidate models are plausible.
Category: Mistakes
What is a limitation of Bayesian Information Criterion?
BIC assumes the true model is among candidates and relies on large-sample approximations. With small datasets, results can be unstable.
Category: Risk
How does sample size affect Bayesian Information Criterion?
The penalty term grows with the logarithm of sample size, increasingly favoring simpler models as more data are available.
Category: Example
What is a competing criterion to Bayesian Information Criterion?
Akaike Information Criterion is the classic alternative. It generally favors more complex models than BIC due to a lighter penalty.
Category: Alternatives