DataTerminology.comNEW
An optimization that applies filters early within storage or connectors to reduce scanned data and improve query performance.
DESCRIPTION
Predicate pushdown moves filtering from the query engine into the storage layer or source system, minimizing data read and transfer. Columnar formats like Parquet and ORC support page and row group filtering. Connectors in engines such as Trino and Spark push eligible predicates to databases and object stores. Benefits include lower I/O, memory, and latency. It complements partition pruning and projection pushdown. Not all expressions are pushable; simplification and sargability help.
ALTERNATIVE DEFINITIONS
Trino
Predicate pushdown allows Trino to push down filtering to connectors to avoid reading data that is not required for a query.
Directly connects pushdown to connector-level filtering and reduced data reads.
Apache Spark
Spark SQL supports Parquet filter pushdown to reduce the amount of data read.
Highlights Parquet-specific filter pushdown in Spark SQL.
RELATED TERMS
Projection Pushdown Columnar Storage Parquet ORC Query Optimization Data Skipping Apache Spark SQL Trino
COMPETING TERMS
KNOWLEDGE AREAS
Data Engineering Data Storage & Operations Real-time & Streaming Data
TARGET AUDIENCE
Data Engineer Data Architect Cloud Architect BI Specialist Software Developer
EXTERNAL SOURCES
- Wikipedia - Query optimization
- Trino - Engine optimizer docs
- Apache Spark - SQL performance tuning
- Apache Parquet - Parquet statistics filtering
TOP QUESTIONS
What is Predicate Pushdown in data processing?
It’s pushing filters into the storage or source system to avoid scanning unnecessary data.
Category: Definition
How does Predicate Pushdown improve performance?
By reducing I/O and data transfer, it lowers latency and resource usage.
Category: Overview
Which file formats support Predicate Pushdown?
Columnar formats like Parquet and ORC use statistics to skip pages and row groups.
Category: Requirements
What expressions are eligible for Predicate Pushdown?
Simple comparisons on partitioned or indexed columns; complex functions may not be pushable.
Category: Mistakes
How does Predicate Pushdown relate to Partition Pruning?
Partition pruning skips entire partitions; pushdown filters within files or tables.
Category: Misunderstanding
What are common mistakes with Predicate Pushdown?
Non-SARGable predicates, uncompressed formats, or missing statistics reduce pushdown effectiveness.
Category: Mistakes
Which engines implement Predicate Pushdown?
Spark SQL, Trino, Presto, and many database connectors support pushdown optimizations.
Category: Overview
How can I test Predicate Pushdown?
Inspect query plans, enable logging, and use explain tools to verify pushed filters.
Category: Requirements
Does Predicate Pushdown help with streaming systems?
Yes, when sources or storage support filter capabilities at ingestion or read time.
Category: Opinion