DataTerminology.comNEW
A Bayesian nonparametric model defining distributions over functions where any finite set of evaluations follows a multivariate normal distribution.
DESCRIPTION
Gaussian processes (GPs) provide flexible, probabilistic modeling for regression and classification with principled uncertainty estimates. They define function priors via mean and covariance (kernel) functions, enabling smoothness and similarity assumptions. Inference uses marginal likelihood to select hyperparameters and yields predictive distributions. While exact GPs scale cubically with data, sparse and approximate methods extend applicability. GPs are widely used in Bayesian optimization, geostatistics (kriging), and active learning.
ALTERNATIVE DEFINITIONS
Wikipedia
In probability theory and statistics, a Gaussian process is a stochastic process in which any finite collection of random variables has a multivariate normal distribution.
Formal stochastic-process definition emphasizing finite-dimensional normality.
GPML (Rasmussen & Williams)
Gaussian processes are distributions over functions, fully specified by a mean function and a covariance function.
Highlights function-space perspective and kernel specification.
RELATED TERMS
Kernel Methods GP Regression Bayesian Optimization Covariance Function Kriging Predictive Uncertainty Nonparametric Bayesian Radial Basis Function Marginal Likelihood Cholesky Decomposition
COMPETING TERMS
Neural Network Support Vector Regression
KNOWLEDGE AREAS
TARGET AUDIENCE
Data Scientist AI Engineer MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Gaussian process
- scikit-learn - User guide to Gaussian processes
- GPyTorch - GPyTorch library for scalable GPs
- BoTorch - Bayesian optimization with GPs
TOP QUESTIONS
What are Gaussian Processes in machine learning?
They are distributions over functions used for regression and classification, providing predictive means and variances with kernels encoding similarity.
Category: Definition
Why use Gaussian Processes?
GPs provide principled uncertainty estimates and flexible prior assumptions. They excel in low- to medium-data regimes and active decision-making.
Category: Relevancy
What are common GP kernels?
Squared Exponential (RBF), Matérn, Rational Quadratic, Periodic, and Linear kernels capture different smoothness and structure assumptions.
Category: Concept
What is the main limitation of Gaussian Processes?
Exact inference scales as O(n^3) with data size due to matrix factorizations. Sparse approximations and inducing points alleviate this.
Category: Risk
How are GP hyperparameters selected?
Maximize the marginal likelihood or use cross-validation and priors. Proper initialization and constraints improve stability.
Category: Process
What is kriging in relation to GPs?
Kriging is GP regression applied in geostatistics, modeling spatial data with covariance structures for interpolation and uncertainty.
Category: Comparison
How do GPs compare to neural networks?
GPs are probabilistic and nonparametric with uncertainty measures, while neural networks are parametric and often scale better to large datasets.
Category: Comparison
Where are Gaussian Processes commonly applied?
Bayesian optimization, geostatistics, time-series interpolation, and active learning benefit from GP uncertainty and sample efficiency.
Category: Usage
Where to learn Gaussian Processes?
Consult the GPML book and scikit-learn, GPyTorch, and BoTorch documentation for theory and implementation.
Category: GettingStarted