DataTerminology.comNEW
Also known as: Record Linkage, Data Matching
The process of identifying and linking records that refer to the same real-world entity across or within data sources.
DESCRIPTION
Entity resolution reconciles duplicates and fragments to create unified views of people, products, or organizations. It uses deterministic rules, probabilistic matching, and machine learning to compare attributes and infer matches. Techniques include blocking, indexing, similarity scoring, and clerical review. It underpins MDM, customer 360, and compliance reporting. Challenges include missing values, inconsistent formats, and privacy constraints. Quality hinges on training data, thresholds, and match explainability. Governance ensures lineage and auditability of merges and survivorship rules. Scalable pipelines employ distributed processing and approximate nearest neighbors. Proper resolution reduces errors, improves analytics, and enables interoperability.
ALTERNATIVE DEFINITIONS
Wikipedia
Record linkage is the task of finding records in a data set that refer to the same entity across different data sources.
Emphasizes cross-source identification of same entities.
IBM
Entity resolution is the process of identifying and linking records that refer to the same entity, even when the records are from different sources or use different formats.
Highlights cross-source heterogeneity and linking process.
AWS
AWS Entity Resolution helps match and link related records stored across multiple applications, channels, and data stores.
Focuses on practical matching and linking of customer records across systems.
RELATED TERMS
Data Deduplication Identity Resolution Weighted Approximate Matching Blocking Master Data Management Data Cleaning Unique Identifier Reference Data
COMPETING TERMS
Deterministic Matching
KNOWLEDGE AREAS
Data Management Data Quality Data Integration & Interoperability Reference & Master Data
TARGET AUDIENCE
Data Steward Data Engineer Data Analyst Data Architect Data Governance Manager
EXTERNAL SOURCES
- Wikipedia - Record linkage
- Dedupe - Open-source entity resolution library docs
- Apache Spark - ANN feature for scalable matching
- OpenRefine - Clustering methods for deduplication
TOP QUESTIONS
What is Entity Resolution?
It identifies and links duplicate or related records that represent the same real-world entity across datasets, enabling unified views like customer 360.
Category: Definition
Why is Entity Resolution important?
Accurate resolution improves analytics, reduces duplication costs, and ensures compliance by consolidating fragmented data into authoritative records.
Category: Relevancy
How is Entity Resolution typically performed?
Using deterministic rules, probabilistic models, and ML to compare attributes, score similarities, and decide matches with thresholds and manual review.
Category: Overview
What are common mistakes in Entity Resolution?
Ignoring data standardization, setting thresholds poorly, and not monitoring drift lead to high false matches or missed links.
Category: Mistakes
What risks exist in Entity Resolution?
Mismatches can create privacy or compliance issues; incorrect merges harm trust and analytics quality.
Category: Risk
How do you evaluate Entity Resolution performance?
Use precision, recall, F1, clerical review, and golden datasets; monitor pair quality over time.
Category: Testing
What techniques scale Entity Resolution?
Blocking keys, locality-sensitive hashing, approximate nearest neighbors, and distributed computation reduce comparisons and enable large-scale matching.
Category: Requirements
Is Entity Resolution part of MDM?
Yes. ER is a core capability in MDM to unify records and apply survivorship rules for authoritative master data.
Category: Opinion
What is a practical analogy for Entity Resolution?
It’s like merging duplicate contacts in your phone by recognizing different entries refer to the same person.
Category: Example