DataTerminology.comNEW
Also known as: Prior Probability Shift
A distribution shift where class prior probabilities change between training and deployment while class-conditional distributions remain stable.
DESCRIPTION
Label shift, also known as prior probability shift, occurs when the marginal distribution of labels changes between training and inference, but the conditional distribution of inputs given labels stays the same. This mismatch can degrade model calibration, thresholding, and performance, especially for probabilistic classifiers. Detection methods use predicted label distributions or black-box estimators; correction techniques include importance weighting and recalibration. Label shift is common when prevalence changes over time, such as seasonal demand or epidemiological trends. Monitoring and mitigation are vital in production ML systems.
ALTERNATIVE DEFINITIONS
Wikipedia
In classification, prior probability shift refers to changes in the class prior probabilities between training and test datasets, while the class-conditional distributions remain unchanged.
Explicitly states unchanged conditionals and changed priors.
ArXiv (Lipton et al.)
We study label shift, the change in the distribution of labels between training and test time, and propose methods to detect and correct it with black box predictors.
Focuses on detection and correction using black-box predictors.
ML Cheat Sheet
Label shift occurs when P(y) changes between training and test, but P(x|y) stays the same.
Compact probabilistic statement of label shift.
RELATED TERMS
Dataset Shift Class Imbalance Calibration Importance Weighting Domain Adaptation Distribution Drift Risk Estimation Model Monitoring
COMPETING TERMS
KNOWLEDGE AREAS
AI Data Science MLOps Testing & Validation Observability & Telemetry
TARGET AUDIENCE
Data Scientist MLOps Engineer AI Engineer Chief AI Officer Data Strategy Consultant
EXTERNAL SOURCES
- Wikipedia - Prior probability shift
- ArXiv - Foundational label shift paper
- scikit-learn - Calibration guidance relevant to shift
- Google Developers - Overview of dataset shift types
TOP QUESTIONS
How does Label Shift differ from Covariate Shift?
Label shift changes P(y) while keeping P(x|y) stable; covariate shift changes P(x) but assumes P(y|x) remains the same.
Category: Definition
What causes Label Shift in production systems?
Changing prevalence, seasonality, policy updates, or emerging trends alter class priors over time, impacting model calibration.
Category: Topic
How can I detect Label Shift?
Compare predicted label distributions, use black-box estimators, and track class prevalence with confidence intervals across cohorts.
Category: HowTo
How do you correct for Label Shift?
Apply importance weighting, recalibrate probabilities, and dynamically adjust classification thresholds based on updated priors.
Category: Management
What is the risk of ignoring Label Shift?
You may experience degraded precision/recall, miscalibrated outputs, and poor business decisions due to mismatched class prevalence.
Category: Risks
Is Label Shift the same as Concept Drift?
No. Concept drift changes P(y|x) relationships, while label shift only changes P(y).
Category: Misunderstanding
Which models are most sensitive to Label Shift?
Probabilistic classifiers and threshold-based systems are particularly sensitive due to dependence on calibrated class probabilities.
Category: Opinion
How does Label Shift affect monitoring dashboards?
Monitor class prevalence, metrics per class, calibration curves, and decision thresholds to detect deviations promptly.
Category: Observability
Where can I learn more about Label Shift?
Start with the Lipton et al. arXiv paper, Google’s dataset shift guide, and calibration references in scikit-learn.
Category: HowTo