DataTerminology.comNEW
Also known as: Equalized Error Rates, Separation
A fairness condition requiring equal true and false positive rates across groups.
DESCRIPTION
Equalized odds requires a model to achieve equal true positive and false positive rates across demographic groups. It conditions fairness on the true outcome, aligning error rates rather than selection rates. This often improves perceived fairness in selective contexts but can conflict with calibration and predictive parity. Post-processing methods can adjust thresholds per group to satisfy it. It is widely cited for binary classification fairness.
ALTERNATIVE DEFINITIONS
Hardt, Price, Srebro (Equality of Opportunity in Supervised Learning)
A predictor satisfies equalized odds if it has equal true positive rate and false positive rate across groups.
Original, concise definition from the seminal paper introducing the criterion.
Fairness and Machine Learning (Barocas, Hardt, Narayanan)
Equalized odds requires that a classifier’s prediction be independent of the protected attribute conditional on the true label.
States the conditional independence view underpinning the metric.
Fairlearn
Equalized odds requires equal false positive rates and equal true positive rates across the sensitive groups.
Operationalizes the requirement via error rates across groups.
RELATED TERMS
Equal Opportunity Fairness Metrics ROC Curve Recall False Positive Rate Thresholding Calibration Confusion Matrix
COMPETING TERMS
Demographic Parity Predictive Parity
RELATIONS
- Broader: Odds
KNOWLEDGE AREAS
AI Data Science AI Governance Testing & Validation
TARGET AUDIENCE
Data Scientist AI Engineer MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Equal opportunity (machine learning)
- arXiv - Seminal paper defining criterion
- Fairlearn - Glossary entry and usage
- Fairness and Machine Learning - Textbook chapter coverage
TOP QUESTIONS
When is Equalized Odds preferable over Demographic Parity?
When label-informed fairness is desired, such as selective admissions or lending, equalized odds aligns error rates across groups rather than equalizing selection rates.
Category: Guidance
How is Equalized Odds measured?
Compute group-wise true positive and false positive rates, then compare differences or distances to equality across groups.
Category: How-To
What are common pitfalls with Equalized Odds?
Ignoring confidence intervals, small-sample instability, and overlooking calibration conflicts can lead to misleading conclusions.
Category: Mistakes
Does Equalized Odds guarantee fairness?
No single metric guarantees fairness. It addresses specific disparities but may conflict with other notions like predictive parity or calibration.
Category: Misunderstanding
How do you enforce Equalized Odds?
Use post-processing threshold adjustments per group, constrained optimization during training, or reweighting techniques to align error rates.
Category: How-To
What is the cost of enforcing Equalized Odds?
It may reduce overall accuracy or utility and complicate deployment with group-specific thresholds, requiring governance and monitoring.
Category: Risks
How does Equalized Odds relate to Equal Opportunity?
Equal opportunity is a relaxation of equalized odds that equalizes only true positive rates for the positive class.
Category: Comparison
How stable is Equalized Odds under drift?
Shifts in base rates or label noise can upset error-rate balance; monitor metrics continuously and adjust as needed.
Category: Monitoring
What tools support Equalized Odds evaluation?
Fairlearn and AIF360 provide functions to compute error-rate disparities and implement post-processing mitigation.
Category: Tooling