DataTerminology.comNEW
Function of model parameters that quantifies how plausible they are given observed data.
DESCRIPTION
In statistical modeling, the likelihood function evaluates parameter values based on observed data, underpinning parameter estimation and hypothesis testing. Maximizing it yields maximum likelihood estimates, widely used for their asymptotic properties. Likelihood contrasts with probability by conditioning on observed data rather than parameters. Its log-transform simplifies optimization and inference via additive properties. Likelihood ratio tests and information criteria compare models. In Bayesian analysis, likelihood combines with a prior to form the posterior.
ALTERNATIVE DEFINITIONS
Wikipedia
In statistics, the likelihood function (often simply the likelihood) is a function of the parameters of a statistical model, given specific observed data.
Clarifies conditioning on observed data with parameters as variables.
NIST Engineering Statistics Handbook
The likelihood function is the joint probability density function viewed as a function of the unknown parameter(s), with the data treated as fixed.
Emphasizes role inversion of parameters and data in likelihood.
RELATED TERMS
Maximum Likelihood Estimation Log-Likelihood Prior Probability Bayesian Inference Likelihood Ratio Test Fisher Information Akaike Information Criterion Deviance Estimator
COMPETING TERMS
RELATIONS
- Narrower: Maximum Likelihood
KNOWLEDGE AREAS
TARGET AUDIENCE
Data Scientist AI Engineer MLOps Engineer Information Analyst
EXTERNAL SOURCES
- Wikipedia - Likelihood function
- NIST - Authoritative explanation and examples
- Stanford Encyclopedia of Philosophy - Philosophical treatment of likelihood
- Statlect - Tutorial with formulas and intuition
TOP QUESTIONS
What does likelihood mean in statistics?
Likelihood evaluates how plausible parameter values are given observed data. It’s central to estimation and hypothesis testing, differing from probability by conditioning on data instead of parameters.
Category: Basics
How is likelihood different from probability?
Probability treats parameters as fixed and data as random; likelihood treats data as fixed and parameters as variables. This role reversal shifts how we infer and optimize models.
Category: Misunderstanding
Why is log-likelihood commonly used?
Taking logs turns products into sums, improving numerical stability and enabling efficient optimization. It also aligns with many asymptotic results like the Fisher information.
Category: Practice
What is Maximum Likelihood Estimation (MLE)?
MLE chooses parameter values that maximize the likelihood function, often yielding estimators with desirable large-sample properties like consistency and asymptotic normality.
Category: Related
How does likelihood relate to Bayesian inference?
In Bayes’ theorem, the posterior is proportional to prior times likelihood. Likelihood encodes data evidence, while the prior encodes beliefs before observing data.
Category: Concept
What are common pitfalls when using likelihood?
Overfitting with overly flexible models and numerical underflow with tiny probabilities are common. Regularization or penalized likelihoods can mitigate these issues.
Category: Mistakes
How do likelihood ratio tests use likelihood?
They compare nested models by the ratio of maximized likelihoods, producing a statistic with asymptotic chi-square distribution under the null hypothesis.
Category: Testing
Is likelihood meaningful without a fully specified model?
No. Likelihood requires a statistical model specifying how data depend on parameters; incomplete models lead to ambiguous or misleading likelihoods.
Category: Requirements
What is a practical example of likelihood in machine learning?
Training logistic regression by maximizing the Bernoulli likelihood (or minimizing log loss) is a direct application of likelihood-based inference.
Category: Example