DataTerminology.comNEW
Also known as: GMM
A probabilistic model representing data as a weighted sum of multiple Gaussian distributions for clustering and density estimation.
DESCRIPTION
A Gaussian Mixture Model (GMM) assumes data arise from a finite mixture of Gaussian components with unknown means and covariances. It supports soft clustering by estimating membership probabilities. Parameters are typically learned via Expectation-Maximization (EM) or variational methods. Model selection commonly uses BIC or AIC to choose the number of components. GMMs capture elliptical clusters and multimodal densities better than k-means. They underpin anomaly detection, speaker verification, and image segmentation.
ALTERNATIVE DEFINITIONS
Wikipedia
A Gaussian mixture model (GMM) is a probabilistic model that assumes all the data points are generated from a mixture of a finite number of Gaussian distributions with unknown parameters.
Emphasizes finite mixture with unknown parameters.
scikit-learn
Gaussian mixture models are probabilistic models that assume that the data are generated from a mixture of a finite number of Gaussian distributions with unknown parameters.
Aligns definition with practical API and usage in Python.
RELATED TERMS
Expectation-Maximization Latent Variable Model Bayesian Mixture Model Covariance Matrix Cluster Analysis Model Selection Bayesian Information Criterion Akaike Information Criterion Soft Clustering
COMPETING TERMS
K-Means Bayesian Gaussian Mixture
RELATIONS
- Broader: Model
KNOWLEDGE AREAS
TARGET AUDIENCE
Data Scientist AI Engineer MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Gaussian mixture model
- scikit-learn - API and usage for GMM
- Stanford University - Lecture notes on EM and GMM
TOP QUESTIONS
What is a Gaussian Mixture Model used for?
GMMs are used for density estimation, soft clustering, and anomaly detection by modeling data as a mixture of Gaussian components.
Category: Overview
How are GMM parameters typically learned?
The Expectation-Maximization algorithm iteratively estimates component assignments and updates means, covariances, and weights until convergence.
Category: Methods
How do GMMs compare to k-means?
Unlike k-means’ hard assignments and spherical assumption, GMMs provide soft probabilities and can model elliptical clusters via covariance matrices.
Category: Comparison
How do I choose the number of components in a GMM?
Use criteria like BIC or AIC, cross-validation, or domain knowledge to balance fit and complexity.
Category: Modeling
When does a GMM perform poorly?
GMMs struggle with non-Gaussian cluster shapes, high-dimensional sparse data, and severe outliers without robustification.
Category: Limits
What covariance types can GMMs use?
Common options include full, tied, diagonal, and spherical covariances, trading flexibility for robustness and computational cost.
Category: Configuration
How do I initialize a GMM?
K-means initialization for means, uniform or data-driven weights, and identity or empirical covariances are standard choices.
Category: Practical
What is a Bayesian Gaussian Mixture?
It places priors on mixture parameters, enabling automatic complexity control and improved regularization under limited data.
Category: Advanced
How do I evaluate a GMM?
Use log-likelihood on validation data, BIC/AIC for model selection, and visualization of responsibilities or contours.
Category: Evaluation