DataTerminology.comNEW
Also known as: Hypothesis Test
A formal procedure that evaluates evidence in data against a null hypothesis using a test statistic and decision rule.
DESCRIPTION
A statistical test assesses whether observed data are consistent with a null hypothesis or favor an alternative. It involves a test statistic, a sampling distribution, and a significance level. P-values quantify extremeness of results under the null; power reflects sensitivity to detect effects. Tests include t-tests, chi-square, ANOVA, and nonparametric methods. Proper multiple testing control and pre-registration help avoid false discoveries.
ALTERNATIVE DEFINITIONS
Wikipedia
A statistical hypothesis test is a method of statistical inference used to decide whether the data at hand sufficiently support a particular hypothesis.
Emphasizes decision-making about hypotheses based on data.
Penn State Statistics
Hypothesis testing is a statistical procedure to decide whether the data provide sufficient evidence to reject a null hypothesis.
Highlights the reject/retain decision framework.
NIST/SEMATECH e-Handbook of Statistical Methods
Hypothesis testing is the process of using statistics to determine the probability that a specific hypothesis is true.
Frames testing in terms of probabilities associated with hypotheses.
RELATED TERMS
Null Hypothesis Alternative Hypothesis p-value Significance Level Test Statistic Power Type I Error Type II Error t-test Chi-square Test
COMPETING TERMS
Estimation Bayesian Inference
RELATIONS
- Narrower: Chi-square Test
KNOWLEDGE AREAS
Mathematics Data Science Testing & Validation
TARGET AUDIENCE
Data Scientist Data Analyst BI Specialist AI Product Manager
EXTERNAL SOURCES
- Wikipedia - Statistical hypothesis testing
- Penn State Statistics - Concise academic overview
- NIST - Handbook chapter on hypothesis tests
- BMJ - Practical guidance on p-values
TOP QUESTIONS
How do I choose the right Statistical Test?
Match the test to your data type, distribution assumptions, number of groups, and design (paired vs. independent). Decision trees or flowcharts help pick t-test, ANOVA, chi-square, or nonparametric tests.
Category: Requirements
What is the biggest misunderstanding about the p-value in a Statistical Test?
A p-value is not the probability the null is true. It is the probability of observing data at least as extreme as the sample under the null hypothesis.
Category: Misunderstanding
What are common mistakes in running a Statistical Test?
Violating assumptions, multiple testing without correction, p-hacking, and selective reporting are pervasive errors that inflate false positives.
Category: Mistakes
What is statistical power in a Statistical Test?
Power is the probability of correctly rejecting a false null. Adequate sample size and effect size considerations are crucial to achieve acceptable power.
Category: Definition
How does multiple testing affect Statistical Tests?
Multiple comparisons increase false discovery rates; control them via Bonferroni, Holm, or FDR procedures like Benjamini–Hochberg.
Category: Risk
Should I use a one-tailed or two-tailed Statistical Test?
Use one-tailed only when a directional effect is justified a priori; otherwise, prefer two-tailed tests to avoid bias and preserve integrity.
Category: Opinion
What is an example of a nonparametric Statistical Test?
The Mann–Whitney U test compares two independent groups without assuming normality. It’s useful for ordinal data or skewed distributions.
Category: Example
How do confidence intervals complement Statistical Tests?
They provide effect size estimates and uncertainty, offering more information than a binary reject/retain decision.
Category: Example
Where can I learn to implement Statistical Tests in software?
Start with R’s stats package or Python’s SciPy, consulting their documentation and reproducible examples using real datasets.
Category: Learning