⏱️Data Freshness

NEW

Also known as: Freshness SLA

Measure of how up-to-date data is relative to its source and defined SLAs.

DESCRIPTION

Data freshness quantifies the age of data and whether it meets recency expectations for analytics or operations. Teams define freshness SLAs and monitor lag between source changes and availability in downstream models. Tools like dbt provide source freshness checks, and orchestration platforms track SLA misses. Freshness directly impacts trust, time-to-insight, and real-time decision-making.

ALTERNATIVE DEFINITIONS

dbt Documentation

Source freshness checks help you ensure your source data is arriving within an acceptable time window, based on your expectations.

Focuses on automated checks against time windows for sources.

Source

Monte Carlo Data

Data freshness refers to how current your data is and whether it’s arriving on time, relative to expectations and SLAs.

Links freshness to SLAs and reliability in data observability.

Source

RELATED TERMS

SLA Service Level Objective Streaming Change Data Capture Incremental Refresh Airflow SLA Monitoring dbt Source Freshness

COMPETING TERMS

Data Timeliness Data Latency

KNOWLEDGE AREAS

Data Quality Observability & Telemetry Data Engineering Real-time & Streaming Data Data Warehousing & BI

TARGET AUDIENCE

Data Engineer Analytics Engineer BI Specialist Data Analyst Data Steward MLOps Engineer

EXTERNAL SOURCES

• Wikipedia - Data quality
• dbt Labs - Automated freshness checks
• Apache Airflow - Task SLA concepts
• Google Cloud - Loading patterns affecting freshness

TOP QUESTIONS

What is Data Freshness and why does it matter?

Data Freshness is recency of data relative to source changes. It matters because stale data leads to outdated insights and missed operational events.

Category: Relevancy

How do you measure Data Freshness?

Measure lag between source update timestamps and downstream availability, benchmarked against defined SLAs or SLOs.

Category: Requirements

What tools help monitor Data Freshness?

dbt source freshness, Airflow SLAs, and observability platforms can track delays and alert on breaches.

Category: Overview

What are common mistakes with Data Freshness?

Ignoring source update schedules, missing timezone handling, and not tracking end-to-end latency cause false alarms or blind spots.

Category: Mistakes

How does streaming impact Data Freshness?

Streaming reduces end-to-end latency, improving freshness, but requires robust backpressure and checkpointing.

Category: Example

What risks arise from poor Data Freshness?

Decisions based on outdated data, broken SLAs, and degraded trust in analytics can occur.

Category: Risk

Is Data Freshness the same as Data Timeliness?

No. Freshness is recency of data; timeliness is availability when needed. They’re related but distinct.

Category: Distinction

How to improve Data Freshness?

Adopt CDC, incremental loads, parallelism, and optimize scheduling. Monitor and resolve bottlenecks quickly.

Category: Management

Where should Data Freshness SLAs be documented?

Document them in data contracts, runbooks, and catalog entries so stakeholders know expectations.

Category: Best practices