DataTerminology.comNEW
A function of model parameters proportional to the probability of observed data under those parameters.
DESCRIPTION
The likelihood function evaluates how plausible parameter values are given observed data and a specified statistical model. In frequentist inference, it underlies maximum likelihood estimation and likelihood ratio tests. In Bayesian inference, the likelihood updates priors to posteriors. Likelihoods are defined up to proportionality constants with respect to parameters. Their curvature informs uncertainty through the Fisher information. Careful model specification and identifiability are essential for meaningful likelihood-based inference.
ALTERNATIVE DEFINITIONS
Wikipedia
In statistics, the likelihood function (often simply the likelihood) of a statistical model is a function of the parameters of the model given specific observed data.
Emphasizes dependence on parameters for fixed observed data.
Statlect
The likelihood function is a function of the parameter that measures how likely it is to observe the sample that has been observed.
Connects likelihood to observed sample plausibility.
NIST/SEMATECH e-Handbook
The likelihood function is the joint density viewed as a function of the parameters with the data fixed.
Defines likelihood via joint density reinterpreted as a function of parameters.
RELATED TERMS
Bayesian Inference Maximum Likelihood Estimation Prior Distribution Posterior Score Function Fisher Information Evidence Lower Bound
COMPETING TERMS
KNOWLEDGE AREAS
TARGET AUDIENCE
EXTERNAL SOURCES
- Wikipedia - Likelihood function
- NIST - Handbook section on likelihood
- Penn State - Course notes on likelihood
- Statlect - In-depth explanation
TOP QUESTIONS
How is a Likelihood Function different from a probability?
Probability treats parameters as fixed and varies data; likelihood treats data as fixed and varies parameters. Likelihoods are defined up to proportionality constants.
Category: Misunderstanding
What is Maximum Likelihood Estimation in the context of the Likelihood Function?
It selects parameter values that maximize the likelihood, often with asymptotic optimality under regularity conditions.
Category: Concept
How does the Likelihood Function appear in Bayesian inference?
It multiplies the prior to yield the posterior (up to normalization), quantifying the data’s influence on beliefs.
Category: Concept
What role does the Likelihood Function play in model comparison?
Likelihood ratios underpin hypothesis tests and information criteria; marginal likelihood integrates the likelihood over priors in Bayesian model comparison.
Category: Evaluation
What are common pitfalls when using the Likelihood Function?
Model misspecification, non-identifiability, and numerical underflow can mislead inference. Use diagnostics and log-likelihood computations.
Category: Mistakes
Why use the log-likelihood instead of the Likelihood Function?
The log-likelihood improves numerical stability and converts products into sums, simplifying optimization and derivation of the score.
Category: Best Practices
How do constraints affect the Likelihood Function?
Constraints restrict parameter space and can change optima and curvature; use constrained optimization or reparameterizations.
Category: Implementation
What is Fisher information in relation to the Likelihood Function?
It is the expected curvature of the log-likelihood, inversely related to estimator variance under regularity conditions.
Category: Concept
Can different models yield the same Likelihood Function?
Yes, non-identifiable or over-parameterized models can induce identical likelihoods, requiring additional constraints or priors.
Category: Risks