🔌Source System

NEW

Also known as: Origin System

Original operational system where data is produced and from which it is extracted for downstream use.

DESCRIPTION

A source system is any operational application, database, or service that generates data for analytical or downstream processing. It includes ERPs, CRMs, logs, APIs, and event streams. Integration teams connect to source systems via connectors, CDC, or batch exports, considering schema changes, SLAs, and data quality. Governance ensures consent, lineage, and compliance. Stable interfaces, contracts, and monitoring help avoid breakages and ensure reliable ingestion.

ALTERNATIVE DEFINITIONS

SAP Help Portal

A source system is a system that supplies data to BW.

Frames the term in the SAP BW context as the data supplier.

Source

Informatica

A source is a table or file from which the data is extracted.

Defines source in ETL tooling context focusing on extract origins.

Source

RELATED TERMS

Extract-Transform-Load ELT Data Ingestion Change Data Capture API Data Lake Data Warehouse Pipelines Middleware System of Record

COMPETING TERMS

Sink Data Store

KNOWLEDGE AREAS

Data Integration & Interoperability Data Engineering Data Management Data Architecture

TARGET AUDIENCE

Data Engineer Data Architect Data Steward Analytics Engineer BI Specialist

EXTERNAL SOURCES

• Wikipedia - Data source
• Microsoft Learn - Data ingestion from diverse source systems
• AWS - Popular data stores as sources for ETL
• Google Cloud - Architectural patterns for ingesting sources

TOP QUESTIONS

Why is the Source System important in data pipelines?

It defines origin, cadence, and quality of ingested data. Understanding its constraints reduces ingestion failures and downstream inconsistencies.

Category: Relevancy

How is a Source System typically integrated?

Through APIs, JDBC/ODBC, file drops, CDC, or event streams. Choice depends on latency, load, schema evolution, and reliability requirements.

Category: Requirements

What are common mistakes with Source Systems?

Ignoring schema changes, underestimating load, and lacking data contracts. Weak monitoring and access controls cause outages and breaches.

Category: Mistakes

How do SLAs apply to a Source System?

SLAs specify availability, latency, and delivery windows. Violations impact downstream analytics and require escalation or buffering strategies.

Category: Management

How is data quality managed at the Source System?

Profiling, validation rules, and lineage monitor quality. Feedback loops to owners fix issues at the source.

Category: Management

What risks accompany integrating a Source System?

Breaking changes, throttling limits, and security constraints. Poorly handled PII can trigger compliance violations.

Category: Risk

What is a competing term for Source System?

Target System is the downstream destination receiving integrated data, often a warehouse or lake.

Category: Alternatives

How does change data capture relate to a Source System?

CDC streams incremental changes, reducing load and latency. It requires reliable keys and consistent ordering.

Category: Example

Where to start learning about Source Systems?

Study ingestion patterns, data contracts, and CDC. Explore vendor docs for connectors and APIs.

Category: Getting-Started