DataTerminology.comNEW
End-to-end tracking of data’s origin, movement, and transformations across systems.
DESCRIPTION
Data lineage documents where data comes from, how it changes, and where it goes. It underpins trust, reproducibility, and regulatory compliance by revealing dependencies and change impacts. Lineage supports root-cause analysis for data quality issues and accelerates incident resolution. It enables impact assessment for schema changes and informs governance and access controls. Modern platforms automate lineage capture via query parsing, logs, and runtime hooks. Standards like OpenLineage promote interoperability across tools. Lineage can be horizontal (across systems) and vertical (within models and reports). Visual lineage graphs aid discovery and stewardship.
ALTERNATIVE DEFINITIONS
Wikipedia
Data lineage includes the data’s origins, what happens to it, and where it moves over time.
Concise emphasis on origin, transformation, and movement over time.
Google Cloud
Data lineage is the ability to track data as it flows from source to destination through pipelines and transformations.
Highlights practical tracking across pipelines and transformations in cloud ecosystems.
OpenLineage
OpenLineage is an open standard for metadata and lineage collection designed to instrument jobs as they are running.
Defines a standard and runtime-oriented approach to lineage collection.
RELATED TERMS
Metadata Management Data Lineage Data Discovery Platform Governance OpenLineage Apache Atlas Data Quality Pipelines Data Lineage Graph
COMPETING TERMS
Data Observability
RELATIONS
- Narrower: Data Lineage
KNOWLEDGE AREAS
Data Management Metadata Management Data Governance Data Architecture Observability & Telemetry Data Engineering
TARGET AUDIENCE
Data Engineer Data Steward Data Architect Data Governance Manager Analytics Engineer BI Specialist MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Data lineage
- OpenLineage - Open standard for lineage metadata
- Apache Atlas - Apache governance and lineage platform
- Microsoft - Purview data lineage concepts
- Google Cloud - Data Catalog lineage guidance
TOP QUESTIONS
Why is Lineage critical for data governance?
Lineage exposes where data originates and how it changes, enabling accountability, auditability, and policy enforcement. It supports compliance, access control decisions, and evidence for regulatory reporting.
Category: Relevancy
How does Lineage help impact assessment?
It maps dependencies between sources, transformations, and consumers, revealing which datasets, dashboards, or models break when upstream changes occur.
Category: Requirements
What is the difference between Lineage and Data Provenance?
Lineage focuses on data flow and transformations across systems, while provenance emphasizes detailed historical context and derivation at granular data-item or research artifact levels.
Category: Misunderstanding
How is Lineage captured in modern stacks?
Tools parse SQL, intercept orchestration metadata, or use runtime hooks and standards like OpenLineage to automatically collect lineage events.
Category: Example
What are common mistakes around Lineage?
Relying on manual diagrams, ignoring Ad-hoc scripts, and not integrating lineage into CI/CD pipelines often leads to stale, incomplete lineage.
Category: Mistakes
How does Lineage support data quality?
By enabling root-cause analysis of anomalies and tracing bad metrics back to specific upstream jobs, columns, or source systems for targeted fixes.
Category: Example
Where to start implementing Lineage?
Start with critical pipelines and BI assets, integrate lineage collection in orchestration and modeling tools, and expose visual graphs in your catalog.
Category: Requirements
What is a hot topic in Lineage today?
Open standards, column-level lineage, and integrating lineage signals into observability and data contracts are rapidly evolving.
Category: Topic
What risks exist if Lineage is missing?
Change blasts can break downstream assets silently, compliance evidence is weak, and incident resolution takes longer due to opaque dependencies.
Category: Risks