DataTerminology.comDesigning, building, and operating data pipelines and platforms that move and transform data
Domain: Data Engineering & Integration
Terms (658)
- ๐งฑ ACID
- โ ACID Transactions
- โ๏ธ AI Accelerator
- ๐งพ ANSI SQL
- ๐ API Integration
- ๐งฉ AWS CloudFormation
- ๐งช AWS Glue
- โก AWS Lambda
- ๐ง AWS SageMaker
- ๐งฉ AWS Step Functions
- ๐ฅ Acquire Data Sources for Sharing
- โ๏ธ Active Metadata
- ๐ Adapters
- ๐ค Agentic AI
- ๐ค Agentic Workflows
- ๐งช Aggregation Pipeline
- ๐จ Alerting
- ๐๏ธ Amazon DynamoDB
- ๐ Amazon EMR
- โก Amazon Kinesis
- ๐๏ธ Amazon S3
- ๐จ Amazon SQS
- ๐ง Amazon Web Services
- ๐ ๏ธ Analytics Engineer
- ๐งฐ Analytics Engineering
- ๐ซ Anti Join
- ๐ Apache Airflow
- ๐น Apache Arrow
- ๐ง Apache Beam
- ๐งฑ Apache Cassandra
- ๐ Apache Flink
- ๐๏ธ Apache HBase
- ๐ Apache Hive
- ๐๏ธ Apache Hudi
- ๐ง Apache Iceberg
- ๐งต Apache Kafka
- ๐งญ Apache Mesos
- ๐ฆ Apache ORC
- ๐ท Apache Pig
- ๐ก Apache Pulsar
- โ๏ธ Apache Spark
- โก Apache Spark SQL
- โก Apache Spark Structured Streaming
- ๐ช๏ธ Apache Storm
- ๐ธ๏ธ Apache Tez
- ๐งฉ Apache TinkerPop
- โ Append-Only
- โ Append-Only Log
- ๐งญ Approximate Nearest Neighbor
- ๐ฏ Approximate Nearest Neighbor Search
- ๐งฎ Arithmetic Coding
- ๐งฑ Array
- ๐ Arrow Flight
- ๐ฆ Artifact Management
- ๐ฆ Artifact Registry
- ๐งฑ Artifacts
- ๐จ At-Least-Once Delivery
- ๐ฏ At-Most-Once Delivery
- ๐งฑ Atomicity
- ๐ค Automated Report
- ๐ค Automated Testing
- ๐ค Automation
- ๐ฆ Avro
- โ๏ธ Azure
- ๐๏ธ Azure Blob Storage
- ๐ Azure Cosmos DB
- ๐๏ธ Azure Data Factory
- ๐๏ธ Azure Data Lake Storage
- ๐งช Azure Databricks
- โก Azure Functions
- โ๏ธ Azure Machine Learning
- ๐งฎ Azure Synapse
- โ๏ธ Azure Synapse Analytics
- โ๏ธ Azure Synapse Serverless
- ๐ณ B-Tree
- ๐ฒ B-tree Index
- ๐งช BASE
- ๐งฉ BLOB
- ๐พ BSON
- โฉ๏ธ Backfill
- ๐ก๏ธ Backpressure
- ๐ Ball Tree
- ๐ฆ Batch Inference
- ๐ฆ Batch Ingestion
- ๐ฆ Batch Processing
- ๐ฆ Batching
- ๐ฒ Beautiful Soup
- ๐พ Big Data
- 1๏ธโฃ Binary Encoding
- ๐ Binning
- ๐งน Bloom Filter
- ๐ Boolean Search
- ๐ฅ Bronze Layer
- ๐ชฃ Bucketization
- ๐ก๏ธ Bulkhead Pattern
- ๐งฉ Byzantine Fault Tolerance
- ๐งฐ CRUD
- ๐๏ธ CSV
- ๐ง Cache Eviction
- ๐งน Cache Invalidation
- โก Caching
- ๐ข Cardinality
- ๐ข Cardinality Estimation
- ๐ Change Data Capture
- ๐งฉ Checkpointing
- ๐ฆ Chunking
- ๐ Circuit Breaker Pattern
- ๐งฐ ClearML
- โก ClickHouse
- ๐ณ Closure Table
- ๐ข Cloud Data Warehouse
- ๐๏ธ Clustered Index
- ๐ชณ CockroachDB
- ๐ก Coding Theory
- ๐๏ธ Column-Family Store
- ๐งฎ Columnar Database
- ๐ Columnar Storage
- ๐๏ธ Columnar Store
- ๐ธ๏ธ Common Crawl
- ๐งฉ Common Table Expression
- ๐๏ธ Compression
- ๐ Compression Ratio
- ๐งฎ Computational Nodes
- ๐ Compute Scaling
- โ Concatenate
- ๐ Concurrency
- ๐ค Consensus
- โ Consistency (ACID)
- โ๏ธ Consistency Model
- ๐ Consistent Hashing
- โ๏ธ Constraint Checking
- ๐๏ธ Container Orchestrator
- ๐งฉ Content Addressable Storage
- ๐งช Controlled Change Principle
- ๐บ๏ธ Coordinate Reference System
- ๐ง Cost-Based Optimizer
- ๐งฉ Couchbase
- ๐ Count-Min Sketch
- ๐ Covering Index
- ๐งฑ Create Table As Select
- โฐ Cron
- โ๏ธ Cross Join
- ๐ธ๏ธ Cypher Query Language
- ๐๏ธ DDL
- ๐ ๏ธ Dagster
- ๐งฉ Dask
- ๐ Data API
- ๐ช Data API Layer
- ๐ฐ๏ธ Data Acquisition
- ๐งฑ Data Architecture
- ๐ Data Change Activity Metrics
- ๐งน Data Cleaning
- ๐งผ Data Cleansing Tools
- ๐๏ธ Data Compression
- ๐ Data Contracts
- โฑ๏ธ Data Currency
- ๐งฌ Data Diversity
- ๐ ๏ธ Data Engineer
- ๐ ๏ธ Data Engineering
- ๐งฒ Data Extraction
- โฑ๏ธ Data Freshness
- ๐งฑ Data Infrastructure
- ๐ฅ Data Ingestion
- ๐ Data Ingestion and Consumption Metrics
- ๐ Data Integration
- ๐ Data Integration Tools
- ๐ Data Integrator
- ๐ Data Lake
- ๐๏ธ Data Lakehouse
- ๐งญ Data Lineage
- ๐ Data Locality
- ๐งฎ Data Manipulation Language
- ๐ถ๏ธ Data Masking
- ๐งฉ Data Matching Engines
- ๐ Data Monitoring
- ๐งญ Data Onboarding
- ๐งญ Data Orchestration
- ๐พ Data Persistence
- โก๏ธ Data Pipeline
- ๐งฑ Data Platform
- ๐งผ Data Preprocessing
- ๐ก๏ธ Data Quality Gates
- ๐งฉ Data Quantum
- โ๏ธ Data Reconciliation
- ๐งฏ Data Redundancy
- ๐งฌ Data Replication
- ๐ฌ Data Science
- ๐ฐ๏ธ Data Source
- ๐งฐ Data Stack
- โณ Data Staging
- ๐ Data Streaming
- ๐ Data Throughput
- ๐ก Data Transfer
- ๐ง Data Transformation
- ๐ Data Transportation
- ๐ Data Types
- โ๏ธ Data Validation
- โก Data Velocity
- ๐๏ธ Data Version Control
- ๐๏ธ Data Versioning
- ๐ช๏ธ Data Volatility
- ๐ฆ Data Volume
- ๐๏ธ Data Warehousing
- ๐งน Data Wrangling
- ๐งพ DataFrame
- โ๏ธ DataOps
- ๐๏ธ Database Management System
- ๐งฉ Database Normalization
- ๐ฅ Databricks
- ๐งฑ Databricks Lakehouse Platform
- ๐ค Databricks Machine Learning
- ๐ Date Format
- ๐๏ธ Dead Letter Queue
- โ Deadlock
- ๐ Debezium
- ๐ Debugging
- ๐ฐ Decimal
- ๐งน Deduplication
- ๐งช Deequ
- โฑ๏ธ Delays
- ๐ฃ Delimiter
- ฮ Delta Lake
- ๐ Delta Live Tables
- ๐งฎ Dense Matrix
- ๐งฎ Derived Attribute
- ๐งญ Design Principles
- ๐ค Detokenization
- ๐ Detrending
- ๐ Dictionary
- ๐งฟ Digital Twin
- ๐บ๏ธ Dijkstraโs Algorithm
- ๐ DirectLake
- ๐ Directed Acyclic Graph
- ๐ฆ Discretization
- ๐ Dissimilarity Measure
- ๐ง Distributed Computing
- ๐งญ Distributed Hash Table
- ๐ Distributed Systems
- ๐ณ Docker
- ๐ Document Database
- ๐๏ธ Document Store
- ๐ป Downsampling
- ๐ฆ DuckDB
- ๐พ Durability
- ๐ ELK Stack
- ๐ ELT
- ๐ก๏ธ ELTG
- โก ESP
- ๐ ETL
- ๐ ๏ธ EXPLAIN
- โก Efficiency
- ๐ Elasticity
- ๐ Elasticsearch
- ๐ค Encoding
- ๐๏ธ Enterprise Data Architecture
- ๐งช Environment
- ๐งฉ Erasure Coding
- ๐ Evaluate and Assess Data Sources
- โก Event
- ๐ Event Log
- ๐ Event Stream
- ๐ก Event Streaming
- โฑ๏ธ Event Time
- โณ Eventual Consistency
- ๐ฆ Exactly-Once Delivery
- โ Exactly-Once Processing
- โ Exactly-Once Semantics
- ๐บ๏ธ Explain Plan
- ๐ Exponential Backoff
- ๐ Extract-Transform-Load
- ๐งฐ FAISS
- ๐ค FTP
- ๐งฉ Feast
- ๐ข Feature
- ๏ธโฃ Feature Hashing
- ๐ง Feature Pipeline
- ๐ Feature Registry
- ๐งฉ Feature Store
- ๐ธ๏ธ Federated Query
- ๐ง Field-Programmable Gate Array (FPGA)
- ๐๏ธ Filter
- ๐ Filtering
- ๐ Flat File
- โก๏ธ Flow
- ๐ Flow Variable
- ๐ Frequency Encoding
- ๐ Full Outer Join
- ๐ Full-Text Search
- ๐ Function Calling
- โก Function as a Service
- ๐ก GPS Coordinates
- ๐บ๏ธ GeoJSON
- ๐ข Geohash
- ๐๏ธ Git Large File Storage
- ๐ง GitOps
- ๐ฅ Gold Layer
- ๐๏ธ Google Bigtable
- ๐ค๏ธ Google Cloud
- ๐๏ธ Google Cloud Bigtable
- โก Google Cloud Functions
- ๐ฃ Google Cloud Pub-Sub
- ๐ชฃ Google Cloud Storage
- ๐ Google Cloud Workflows
- ๐ Google Spanner
- ๐ฃ๏ธ Gossip Protocol
- ๐ฌ Granularity
- ๐ธ๏ธ Graph Database
- ๐งฉ Graph Partitioning
- โ Great Expectations
- ๐ธ๏ธ Gremlin
- โฌก H3
- ๐พ HBM
- ๐งฑ HDFS
- ๐ธ๏ธ HNSW
- ๐ Hadoop
- ๏ธโฃ Hash Index
- ๐ Hash Partitioning
- ๐ Hash Table
- ๐ Hashing
- ๐งฉ Hashing Trick
- ๐ Haversine Distance
- ๐งฐ Haystack
- โก High Frequency Data
- ๐ฆ Hinted Handoff
- ๐ฐ๏ธ Historical Versions
- ๐ฒ Huffman Coding
- ๐ Idempotency
- ๐ Implement Data Sharing-Integration Services
- ๐จ Incident Management
- ๐ Incremental Refresh
- ๐ Index
- ๐ Indexing
- โ๏ธ Inference Engine
- ๐ Inference Optimization
- ๐ InfluxDB
- ๐ Information Retrieval
- ๐๏ธ Infrastructure as Code
- โ๏ธ Inner Join
- ๐ข๏ธ InnoDB
- ๐ข Integer Arithmetic
- โ๏ธ Integration Platform as a Service
- ๐ Integration Testing
- ๐๏ธ Inverted File Index
- ๐ Inverted Index
- ๐ Isolation
- ๐ Isolation (ACID)
- ๐งพ JSON
- ๐งฑ JSONFormer
- ๐ธ๏ธ JanusGraph
- โ Java
- ๐ถ Jitter
- ๐ Join
- ๐ณ KD-Tree
- ๐๏ธ KV Cache
- ๐งต Kafka Streams
- ๐ Kappa Architecture
- ๐ Key
- ๐ Key-Value Store
- ๐ธ๏ธ Knowledge Graphing
- โ๏ธ Kubeflow
- โฐ Kubernetes CronJob
- ๐ท๏ธ Label Encoding
- ๐๏ธ Lakehouse Architecture
- ๐ท๏ธ Lakehouse Domain
- ๐งฉ LangChain
- โณ Late Data
- ๐ฐ๏ธ Late Data Handling
- โฑ๏ธ Latency Optimization
- ๐ Latitude-Longitude
- โฌ ๏ธ Left Join
- ๐งญ Lineage
- ๐งฉ Linearizability
- ๐๏ธ List Partitioning
- ๐ฆ LlamaIndex
- ๐งญ Locality-Sensitive Hashing
- ๐ Locking
- ๐ฒ Log-Structured Merge-Tree
- ๐ Long Context
- ๐๏ธ Lossless Compression
- ๐ Lossy Compression
- ๐ MLOps
- ๐บ๏ธ MapReduce
- ๐ฌ MariaDB
- ๐ฒ Materialized Path
- ๐ Medallion Architecture
- ๐ Merge
- ๐ฌ Message Broker
- ๐ฌ Message Queue
- โฑ๏ธ Micro Batch Processing
- โฑ๏ธ Micro-batching
- ๐งต Microsoft Fabric
- ๐ Migration
- ๐๏ธ Milvus
- ๐งฉ MinHash
- ๐ Model Deployment
- โ๏ธ Model Serving
- ๐๏ธ Model Versioning
- โ๏ธ Modern Data Stack
- ๐๏ธ MongoDB
- ๐ก Monitor Data Movement
- ๐ Monitoring
- ๐ฐ๏ธ Monte Carlo Data
- ๐ Multi-Region Replication
- ๐ Multiversion Concurrency Control
- ๐ฌ MySQL
- โ NaN
- โฑ๏ธ Near Real-time
- ๐ธ๏ธ Neo4j
- ๐ณ Nested Set Model
- ๐งฑ NewSQL
- ๐งฑ NewSQL Database
- ๐งฐ NoSQL
- ๐๏ธ NoSQL Database
- ๐ซ Null Handling
- ๐งฎ NumPy Array
- ๐ฏ Numeric Precision
- ๐ค OCR
- ๐ OCR Engine
- ๐๏ธ ORC
- ๐๏ธ Object Storage
- ๐งฉ Object-Relational Mapping
- ๐๏ธ Offline Feature Store
- โก Online Feature Store
- โก Online Inference
- ๐งญ OpenLineage
- ๐๏ธ OpenMetadata
- ๐งญ OpenSearch
- ๐ Operational Metadata
- ๐ Optimistic Concurrency Control
- ๐ข๏ธ Oracle Database
- ๐บ๏ธ Oracle Spatial
- ๐งฉ Orchestration
- ๐๏ธ Out-of-Core Learning
- ๐ Out-of-Order Events
- โ๏ธ Overhead
- ๐๏ธ PGAdmin
- ๐ Pagination
- ๐ผ Pandas
- ๐ผ Pandas DataFrame
- โ๏ธ Parallel Computing
- โก Parallel Processing
- โก Parallelism
- ๐งฑ Parquet
- ๐งฉ Partition
- ๐๏ธ Partition Pruning
- ๐งฉ Partitioning
- โ๏ธ Performance Tuning
- โฐ Periodic Data Refreshment
- ๐พ Persistence
- ๐ฒ Pinecone
- ๐ ๏ธ Pipeline Orchestration
- ๐ Pipelines
- ๐งฉ PolyBase
- ๐๏ธ Polyglot Persistence
- ๐บ๏ธ PostGIS
- ๐ PostgreSQL
- ๐งฐ Power Query
- โ Predicate
- โ๏ธ Predicate Pushdown
- ๐งญ Prefect
- ๐งผ Preprocessing
- โก๏ธ Presto
- ๐ Primary Key
- ๐ Processing Time
- ๐ Product Analytics
- ๐ฆ Product Quantization
- ๐งช Profiling
- ๐ Protocol Buffers
- ๐ฃ PublishโSubscribe
- โ Pydantic
- ๐ Python
- ๐งฉ Quadtree
- ๐ Quantile Binning
- โ๏ธ Quantization
- ๐ Query
- โก Query Caching
- ๐ฃ๏ธ Query Execution Plan
- ๐ Query Federation
- ๐ Query Optimization
- ๐ง Query Optimizer
- ๐งญ Query Planner
- ๐ฌ Queue
- ๐ R
- ๐ R (Programming Language)
- ๐ฒ R-Tree
- ๐๏ธ RDBMS
- ๐งฑ RDD
- ๐ RFC 4180
- โ๏ธ ROCm
- ๐๏ธ Range Partitioning
- ๐ Rank-Based Feature
- ๐ฅ Raw Data
- ๐ Ray
- ๐ Read Committed
- ๐ Read-Your-Writes Consistency
- โก Real-Time
- โก Real-time Analytics
- โก Real-time Data
- ๐ช Recursive CTE
- ๐ Recursive Relationships
- โก Redis
- ๐ ๏ธ Refactoring
- ๐ค Regular Expression
- ๐ Relational Table
- ๐ก๏ธ Reliability
- ๐งฌ Replicate
- ๐ Replication
- ๐ฆ Repository
- ๐งญ Resolver
- ๐งฎ Resource Allocation
- ๐ ๏ธ Resource Utilization Protocols
- ๐ Retry Pattern
- ๐ Retry Policy
- โฉ๏ธ Reverse ETL
- ๐ Reverse Geocoding
- โฉ๏ธ Reversibility
- โก๏ธ Right Join
- ๐ Robust Scaling
- โฉ๏ธ Rollback
- ๐ Rolling Window
- ๐งฉ Row Level Changes
- ๐งฑ Row-Oriented Storage
- ๐ง Rule-Based Optimization
- ๐งฉ Rule-Based Optimizer
- ๐๏ธ Run-Length Encoding
- ๐ S2 Geometry
- ๐งฎ SQL
- ๐๏ธ SQLite
- ๐ง SageMaker Feature Store
- ๐พ Savepoint
- ๐ ScaNN
- ๐ท Scala
- ๐ Scalability
- โฑ๏ธ Scheduler
- ๐ Scheduling
- ๐ Schema Drift
- ๐ก๏ธ Schema Enforcement
- ๐งฉ Schema Evolution
- ๐งฉ Schema Matching
- ๐ Schema Registry
- ๐งฑ Schema Versioning
- ๐งฉ Schema-less
- ๐๏ธ Schema-on-Read
- ๐ท๏ธ Scrapy
- ๐งผ Scrubbing
- ๐ Search
- ๐งญ Search Engine
- ๐ Search Index
- ๐ ๏ธ Self-Service Data Infrastructure
- ๐ Semantic Search
- ๐ Semantic Similarity
- ๐ Semi Join
- ๐งฉ Semi-structured Data
- ๐งฉ Sentence-BERT
- ๐งฑ Serializable Isolation
- ๐ฆ Serialization
- ๐ Series
- โ๏ธ Serverless Computing
- ๐งญ Service Discovery
- ๐ Service Level Indicator
- ๐ฏ Service Level Objective
- ๐ก Service Monitoring
- โฑ๏ธ Session Window
- โ Set Operations
- ๐๏ธ Shapefile
- ๐งฉ Sharding
- ๐ Shuffling
- ๐ฅ Silver Layer
- ๐ฏ Similarity Search
- ๐๏ธ Single Table Design
- ๐ค Sink Data Store
- ๐งฎ Sketching
- ๐ช Sliding Window
- ๐งฑ Smart Open-source Middleware
- โ๏ธ Snowflake
- โ Soda Core
- ๐๏ธ Sorting
- ๐๏ธ Source Coding
- ๐ Source Data Store
- ๐ Source System
- ๐ธ๏ธ Spark GraphX
- ๐ง Spark MLlib
- โก Spark SQL
- ๐ Spark Streaming
- โก Spark Structured Streaming
- ๐งฉ Sparse Matrix
- ๐งฉ Sparse Vector
- 0๏ธโฃ Sparsity
- ๐งญ Spatial Index
- โก Speculative Decoding
- โก๏ธ Spot Instances
- ๐ Standardization
- ๐ง Stateful Processing
- ๐งฉ Stored Procedure
- ๐ Stream Analytics
- โก Stream Processing
- ๐ Streaming
- ๐ง Streaming Data
- โก Streaming Ingestion
- ๐ Streaming Join
- โ Strong Consistency
- ๐๏ธ Structural Changes
- ๐ Structured Streaming
- ๐ Surrogate Key
- โก Synapse Spark Pool
- ๐๏ธ TSV
- โณ TTL
- ๐ Table
- โฐ Task Scheduling
- ๐งฉ Tecton
- ๐๏ธ Temporary Table
- ๐งช Testing
- ๐งน Text Normalization
- โฑ๏ธ Time Series Database
- ๐ฐ๏ธ Time Travel
- ๐ฐ๏ธ Time Travelling
- โณ Timeout
- โฑ๏ธ TimescaleDB
- ๐ Transaction
- ๐งพ Transaction Activity Data
- ๐ Transaction Log
- ๐ง Transactionally Consistent Data Lake
- ๐ง Transformation
- ๐ฒ Tree Traversal
- ๐ฏ Triggers
- ๐ Trino
- ๐ Tumbling Window
- ๐ UUID
- โ Union
- ๐ Unique Constraint
- ๐งช Unit Testing
- โ Unit Tests
- ๐ Upsert
- ๐งฒ Vector Database
- ๐งญ Vector Index
- ๐งญ Vector Search
- ๐งฎ Vectorization
- โก Vectorized Execution
- ๐ข Versioning
- ๐งฉ Vertex AI
- ๐ช View
- ๐งโ๐ป Visual Studio Code
- ๐ฆ Volume
- ๐ง Watermark
- ๐ท๏ธ Web Scraping
- ๐๏ธ Wide-Column Store
- ๐ญ Window Function
- ๐ Windowed Aggregation
- ๐ช Windowing
- ๐ Workflow
- ๐บ๏ธ Workflow Orchestration
- ๐งพ Write-Ahead Logging
- ๐งพ YAML
- ๐งถ YARN
- ๐ง ZeRO
- ๐ Zero Downtime Deployment
- ๐ ๏ธ dbt
- ๐ lakeFS