DataTerminology.comNEW
The probability, under the null hypothesis, of observing results at least as extreme as those actually observed.
DESCRIPTION
A p-value quantifies the extremeness of data assuming the null hypothesis is true. Small p-values suggest observed results are unlikely under the null, informing but not proving conclusions. It does not measure the probability the null is true nor the magnitude of an effect. Proper interpretation requires study design, effect sizes, confidence intervals, and domain knowledge. Multiple testing and p-hacking can invalidate inferences without correction.
ALTERNATIVE DEFINITIONS
UCLA Statistical Consulting (IDRE)
The p-value is the probability under the null hypothesis of obtaining a result equal to or more extreme than what was actually observed.
Clear, concise definition emphasizing conditioning on the null hypothesis.
NIST/SEMATECH e-Handbook of Statistical Methods
The p-value is the smallest level of significance at which the null hypothesis would be rejected for the given data.
Defines p-value via equivalence to significance thresholds.
RELATED TERMS
Null Hypothesis Statistical Significance Confidence Interval Type I Error Type II Error t-test Test Statistic Variance Analysis
COMPETING TERMS
KNOWLEDGE AREAS
Mathematics Data Science Testing & Validation
TARGET AUDIENCE
Data Scientist Data Analyst Information Analyst
EXTERNAL SOURCES
TOP QUESTIONS
What does a small P-value indicate?
It suggests the observed data are unlikely under the null hypothesis, providing evidence against H0 but not proving any hypothesis.
Category: Misunderstanding
Is P-value the probability the null hypothesis is true?
No. It conditions on the null being true and measures data extremeness, not hypothesis probabilities.
Category: Misunderstanding
How should I report P-values?
Report exact p-values along with effect sizes and confidence intervals to convey both significance and magnitude.
Category: Usage
What are common pitfalls with P-values?
p-hacking, multiple comparisons without correction, and equating significance with practical importance are frequent errors.
Category: Mistakes
How do multiple tests affect P-values?
They inflate false positives. Apply corrections (e.g., Bonferroni, Benjamini–Hochberg) to control error rates.
Category: Risks
When should I prefer Bayes Factors over P-values?
When comparing evidential support between models or incorporating priors, Bayes factors offer a Bayesian alternative to NHST.
Category: Alternatives
Why combine P-values with confidence intervals?
Confidence intervals communicate effect size and uncertainty, complementing the binary nature of significance tests.
Category: Relevancy
How do P-values relate to Type I error?
A p-value below alpha leads to rejecting H0, with alpha bounding the long-run Type I error rate under H0.
Category: Topic
Can large samples make tiny P-values for trivial effects?
Yes. With enough data, small effects become statistically significant; always assess practical significance.
Category: Risks