DataTerminology.comNEW
A statistical inference procedure for assessing whether data provide sufficient evidence to reject a specified null hypothesis.
DESCRIPTION
Hypothesis testing evaluates claims by comparing observed data against expectations under a null hypothesis. Test statistics and p-values quantify inconsistency with the null, while power analysis guides sample size. Errors are controlled through significance levels and corrections for multiple testing. Alternative approaches include estimation with confidence intervals and Bayesian methods. Proper pre-registration and avoidance of p-hacking strengthen validity.
ALTERNATIVE DEFINITIONS
Wikipedia
A statistical hypothesis test is a method of statistical inference used to decide whether the data at hand sufficiently support a particular hypothesis.
Focuses on decision-making based on data support.
NIST/SEMATECH
Hypothesis testing is a statistical method that uses sample data to evaluate a hypothesis about a population.
Emphasizes population-based inference using sample data.
RELATED TERMS
Null Hypothesis Alternative Hypothesis p-value Type I Error Type II Error Statistical Power Confidence Interval Test Statistic t-test Variance Analysis
COMPETING TERMS
RELATIONS
KNOWLEDGE AREAS
Mathematics Data Science Testing & Validation
TARGET AUDIENCE
Data Scientist Data Analyst BI Specialist
EXTERNAL SOURCES
- Wikipedia - Statistical hypothesis testing
- NIST/SEMATECH - NIST chapter on hypothesis tests
- Carnegie Mellon University - Lecture notes on hypothesis tests
- OpenIntro Statistics - Open textbook covering tests
TOP QUESTIONS
What are the components of Hypothesis Testing?
Define null and alternative hypotheses, choose a test statistic and significance level, compute a p-value, and decide to reject or not reject the null.
Category: Definition
What does a p-value represent in Hypothesis Testing?
It’s the probability, under the null, of observing data at least as extreme as what was observed. It is not the probability the null is true.
Category: Misunderstanding
How do I control Type I error in Hypothesis Testing?
Set an alpha (e.g., 0.05) and use corrections like Bonferroni or Benjamini–Hochberg for multiple tests.
Category: Best Practice
How do I improve power in Hypothesis Testing?
Increase sample size, reduce variance, choose more sensitive tests, or increase effect size through design.
Category: Requirements
When should I prefer confidence intervals over tests?
When effect size estimation and uncertainty are more informative than a binary decision, report intervals alongside or instead of tests.
Category: Opinion
What’s the difference between one-tailed and two-tailed tests?
One-tailed tests assess effect in one direction; two-tailed test for any difference. Choose based on hypothesis and pre-specify.
Category: Concept
What is the risk of p-hacking in Hypothesis Testing?
Data dredging inflates false positives. Pre-registration and correction for multiple comparisons mitigate this risk.
Category: Risk
How do nonparametric tests relate to Hypothesis Testing?
They relax distributional assumptions (e.g., Wilcoxon, Kruskal–Wallis), useful with ordinal or non-normal data.
Category: Technique
Can Bayesian methods replace Hypothesis Testing?
Bayesian inference offers probability statements about parameters and models; it’s an alternative that complements or replaces frequentist tests depending on goals.
Category: Comparison