DataTerminology.comNEW
Systematic unfairness in data or models that yields discriminatory or inequitable outcomes across individuals or groups.
DESCRIPTION
Bias in AI fairness refers to harmful or unjust patterns embedded in data, models, or processes that disadvantage certain groups. It can arise from sampling, labeling, historical inequities, or design choices. Fairness assessments use metrics like demographic parity, equalized odds, and disparate impact. Mitigation strategies include data rebalancing, algorithmic constraints, post-processing, and human oversight. Governance practices require documentation (model cards), bias impact analysis, and monitoring. Regulation and standards increasingly define expectations for transparency, accountability, and risk management.
ALTERNATIVE DEFINITIONS
NIST AI Risk Management Framework
Bias in AI can lead to harmful or unfair outcomes and may be present in data, models, or human-system components across the AI lifecycle.
Frames bias as a lifecycle risk spanning data, models, and socio-technical context.
OECD.AI Glossary
Bias refers to systematic and unfair discrimination in algorithmic systems, often arising from data, design choices, or deployment context.
Emphasizes systemic unfairness and multiple sources, including deployment.
IBM
AI bias is when an algorithm produces results that are systematically prejudiced due to erroneous assumptions in the machine learning process.
Highlights erroneous assumptions during ML as a driver of systematic prejudice.
RELATED TERMS
AI Fairness Demographic Parity Equalized Odds Disparate Impact Responsible AI Explainability Model Cards Governance Bias Mitigation
COMPETING TERMS
RELATIONS
- Broader: Fairness
KNOWLEDGE AREAS
AI Governance Data Science Data Management Data Privacy & Compliance
TARGET AUDIENCE
AI Engineer Data Scientist MLOps Engineer Data Governance Manager Chief Data Officer AI Product Manager
EXTERNAL SOURCES
- Wikipedia - Algorithmic bias
- NIST - RMF playbook guidance on bias
- Google AI - Responsible AI practices and bias
- arXiv - Survey of algorithmic fairness
TOP QUESTIONS
What causes Bias (Fairness) in AI systems?
Common causes include skewed or incomplete training data, historical discrimination encoded in labels, flawed objectives, and feedback loops during deployment.
Category: Understanding
How is Bias (Fairness) measured in practice?
Teams apply fairness metrics like demographic parity, equalized odds, predictive parity, and disparate impact, then compare across protected groups.
Category: Methods
What are effective mitigation techniques for Bias (Fairness)?
Approaches include reweighting or resampling data, adversarial debiasing, constrained optimization, and post-processing of decisions to satisfy fairness criteria.
Category: BestPractice
Why is Bias (Fairness) a governance issue, not just a technical one?
Bias involves value judgments, legal obligations, and social impact, requiring policy, accountability, and stakeholder engagement beyond algorithms.
Category: Relevancy
How does regulation influence Bias (Fairness) management?
Emerging laws and standards mandate risk assessments, transparency, and human oversight, increasing documentation and monitoring requirements.
Category: Regulation
What is a common misconception about Bias (Fairness)?
That a single metric or technique can make a model universally fair; fairness is contextual and involves trade-offs.
Category: Misunderstanding
How should teams monitor Bias (Fairness) post-deployment?
Implement drift and fairness monitoring on live data, segmenting metrics by groups and auditing decision outcomes regularly.
Category: Operations
What data practices reduce Bias (Fairness) risks?
Diverse sampling, robust labeling guidelines, bias-aware feature engineering, and transparent documentation (datasheets, model cards) are essential.
Category: Requirements
What is a hot topic in Bias (Fairness) today?
Measuring and mitigating bias in large language models and foundation models remains an active research and policy area.
Category: Topic