DataTerminology.comNEW
Resampling technique that partitions data into folds to train and validate models for generalization estimates.
DESCRIPTION
Cross-validation tests model performance by training on subsets of data and validating on complementary subsets. Common variants include k-fold, stratified k-fold for class balance, and leave-one-out for small datasets. It reduces variance compared to single holdout splits and helps detect overfitting. Nested cross-validation provides unbiased estimates during hyperparameter tuning. Careful splitting prevents leakage across folds.
ALTERNATIVE DEFINITIONS
Wikipedia
In statistics, cross-validation is a model validation technique for assessing how the results of a statistical analysis will generalize to an independent data set.
Highlights generalization assessment to independent data.
scikit-learn
The following section describes cross-validation strategies for model evaluation, including KFold and StratifiedKFold, and how to use them to evaluate and tune estimators.
Emphasizes practical strategies and estimator evaluation.
Machine Learning Mastery
Cross-validation is a resampling procedure used to evaluate machine learning models on a limited data sample.
Plain-language definition widely used in practice tutorials.
RELATED TERMS
K-Fold Cross-Validation Stratified Sampling Bootstrap Hyperparameter Tuning Model Overfitting Leakage Nested Cross-Validation Train-Test Split
COMPETING TERMS
Model Validation
RELATIONS
- Narrower: K-Fold Cross-Validation
KNOWLEDGE AREAS
Data Science MLOps Testing & Validation
TARGET AUDIENCE
Data Scientist MLOps Engineer AI Engineer
EXTERNAL SOURCES
- Wikipedia - Cross-validation (statistics)
- scikit-learn - Documentation of CV strategies and APIs
- Keras - Model evaluation tips in Keras workflow
- Penn State STAT 508 - Course notes on cross-validation
TOP QUESTIONS
When should I use K-fold Cross-Validation?
Use k-fold when data is limited and you need a reliable generalization estimate. A common choice is k=5 or 10 for a bias-variance tradeoff.
Category: Guidance
What is Stratified Cross-Validation?
It preserves class proportions in each fold, improving reliability for imbalanced classification problems.
Category: Understanding
What is data leakage in Cross-Validation?
Leakage occurs when information from validation folds influences training, inflating performance. Always fit preprocessing within each training fold.
Category: Risk
How does Cross-Validation help with hyperparameter tuning?
It provides a robust estimate of performance for each configuration, reducing overfitting to a single validation split.
Category: Application
What is Nested Cross-Validation?
An outer CV estimates generalization while an inner CV tunes hyperparameters, preventing optimistic bias.
Category: Technique
Is Leave-One-Out Cross-Validation always better?
No. It has low bias but high variance and can be computationally expensive, making k-fold preferable in many cases.
Category: Misunderstanding
How many folds should I choose?
Common practice is 5 or 10 folds. More folds increase computation and variance; fewer increase bias.
Category: Guidance
Should I shuffle before Cross-Validation?
Yes for i.i.d. data; no for time series where temporal order matters. Use time series split for sequences.
Category: HowTo
What metric should I average across folds?
Use a domain-appropriate metric (e.g., ROC AUC, F1, RMSE) and report mean and variance or confidence intervals.
Category: HowTo