DataTerminology.comNEW
Also known as: Student’s T-Test, Hypothesis Test
Statistical test assessing whether means of two groups differ significantly under assumptions.
DESCRIPTION
A t-test evaluates if the difference between group means is statistically significant given sample variance and size. Common variants include one-sample, independent two-sample, Welch’s, and paired t-tests. Assumptions often include approximate normality and independence, with robustness to moderate deviations. It is widely used in A/B testing, clinical trials, and experimental research. Reporting includes p-values, confidence intervals, and effect sizes. Proper design, power analysis, and multiple testing control are crucial.
ALTERNATIVE DEFINITIONS
Wikipedia
A t-test is any statistical hypothesis test in which the test statistic follows a Student’s t-distribution under the null hypothesis.
Defines t-tests by the distribution of the test statistic.
NIST/SEMATECH e-Handbook of Statistical Methods
The t test is used to determine if the mean of a normally distributed population differs from a specified value.
Focuses on one-sample t-test and normality assumption.
GraphPad
The t test compares the means of two groups.
Practical emphasis on comparing two group means.
RELATED TERMS
Variance Analysis p-value Confidence Interval Normal Distribution Degrees of Freedom Effect Size Two-Sample t-Test Paired t-Test Welch’s t-Test
COMPETING TERMS
KNOWLEDGE AREAS
Mathematics Data Science Testing & Validation
TARGET AUDIENCE
Data Scientist Data Analyst Information Analyst BI Specialist
EXTERNAL SOURCES
- Wikipedia - Student’s t-test
- NIST - One-sample t-test overview
- Penn State - Course notes on t-tests
- GraphPad - Practical t-test guidance
- IBM - Statistics tooling context
TOP QUESTIONS
When should I use a T-Test?
Use t-tests to compare means between two groups or one sample against a known value. Ensure assumptions are met or consider robust alternatives like Welch’s test.
Category: Requirements
What assumptions underlie the T-Test?
Independence, approximate normality, and equal variances for the classic two-sample test. Welch’s t-test relaxes the equal variance assumption.
Category: Requirements
What is a common mistake with T-Tests?
Multiple uncorrected comparisons inflate false positives. Misinterpreting p-values as effect magnitude is another frequent error.
Category: Mistakes
How does T-Test differ from Z-Test?
T-tests use t-distribution with estimated variance; z-tests assume known population variance and typically large samples.
Category: Comparison
How is T-Test used in A/B testing?
It compares conversion or metric means between control and treatment groups, reporting p-values and confidence intervals to infer significance.
Category: Example
What is Welch’s T-Test?
A two-sample t-test variant that does not assume equal variances, providing more reliable results when variances differ.
Category: Definition
How do I report T-Test results?
Include test type, t-statistic, degrees of freedom, p-value, confidence interval, and effect size (e.g., Cohen’s d).
Category: Best practices
What is the biggest misunderstanding about T-Tests?
Believing significance implies practical importance. Statistical significance does not guarantee meaningful effect size.
Category: Misunderstanding
What are robust alternatives to T-Tests?
Mann–Whitney U test for non-normal data, bootstrap-based inference, or permutation tests to relax distributional assumptions.
Category: Alternatives