DataTerminology.comNEW
Unjust differential treatment of individuals or groups, often assessed through fairness metrics in AI systems and decision processes.
DESCRIPTION
Discrimination manifests as unfair treatment based on protected characteristics, with algorithmic systems potentially amplifying harms. In AI, measuring and mitigating discriminatory outcomes requires fairness metrics, audits, and governance. Regulatory frameworks and ethical guidelines demand proactive assessment and remediation. Transparent documentation, stakeholder engagement, and impact monitoring help prevent discriminatory effects.
ALTERNATIVE DEFINITIONS
UN OHCHR
Discrimination is any distinction, exclusion, restriction or preference based on race, colour, descent, or national or ethnic origin which has the purpose or effect of nullifying or impairing the recognition, enjoyment or exercise, on an equal footing, of human rights and fundamental freedoms.
Formal human rights definition emphasizing protected characteristics and effects.
European Commission
Discrimination occurs when someone is treated less favourably than someone else in a comparable situation on grounds such as sex, racial or ethnic origin, religion or belief, disability, age or sexual orientation.
EU policy framing with common protected grounds and comparators.
NIST AI Risk Management Framework
The AI RMF addresses risks including harmful bias and discriminatory outcomes, promoting practices to manage and govern such risks throughout the AI lifecycle.
Contextualizes discrimination within AI risk management practices.
RELATED TERMS
Systematic Error Algorithmic Fairness Equal Opportunity Demographic Parity Protected Attribute Ethical AI AI Governance Impact Assessment
COMPETING TERMS
KNOWLEDGE AREAS
AI Governance Data Privacy & Compliance Data Governance Data Management
TARGET AUDIENCE
Chief AI Officer Data Governance Manager Chief Data Officer Data Privacy Officer AI Product Manager Data Scientist
EXTERNAL SOURCES
- Wikipedia - Discrimination
- OECD AI Principles - Principles addressing fairness and harm
- NIST - AI RMF guidance on bias risks
- ACM FAccT - Research community on fairness
TOP QUESTIONS
Why is Discrimination a critical concern in AI systems?
AI can scale biased decisions, affecting many people; managing discrimination is essential for trust and compliance.
Category: Relevancy
How is Discrimination measured in algorithmic outputs?
Using fairness metrics like demographic parity, equalized odds, and calibration across groups.
Category: Management
What causes Discrimination in data-driven models?
Biased datasets, proxy variables, and feedback loops can introduce or amplify discriminatory outcomes.
Category: Topic
What are common mistakes in mitigating Discrimination?
Applying a single fairness metric universally, ignoring context, and neglecting stakeholder impacts.
Category: Mistakes
What risks arise from ignoring Discrimination in AI?
Legal exposure, reputational damage, and harm to affected communities are likely consequences.
Category: Risks
What are alternatives or counterpoints to Discrimination in AI?
Algorithmic fairness frameworks and non-discrimination policies guide design and auditing.
Category: Alternatives
How should Discrimination be managed across the AI lifecycle?
Incorporate fairness assessments during data collection, modeling, deployment, and monitoring with governance oversight.
Category: Management
What’s a striking analogy for Discrimination in data?
A funhouse mirror: data distortions reflect unfair realities unless corrected by careful design.
Category: Example
Where to start learning about Discrimination and AI fairness?
Study NIST AI RMF, OECD AI Principles, and ACM FAccT proceedings.
Category: Learning