DataTerminology.comNEW
Also known as: HMC
A gradient-based MCMC method that uses Hamiltonian dynamics to efficiently sample from complex posterior distributions.
DESCRIPTION
Hamiltonian Monte Carlo simulates Hamiltonian dynamics to propose distant, coherent moves, reducing random walk behavior and improving sampling efficiency. It leverages gradients of the log posterior to navigate typical sets with high acceptance rates. Widely implemented in probabilistic programming frameworks like Stan and PyMC, HMC excels in high-dimensional models. Tuning step size and trajectory length is crucial; NUTS automates this. HMC underpins modern Bayesian inference workflows across science and industry.
ALTERNATIVE DEFINITIONS
Stan Documentation
Hamiltonian Monte Carlo uses gradients of the log density to generate distant proposals that can be accepted with high probability, reducing random-walk behavior.
Emphasizes gradient information and reduced random walk.
Michael Betancourt (arXiv)
Hamiltonian Monte Carlo leverages Hamiltonian dynamics to generate coherent trajectories through parameter space that efficiently explore typical sets.
Highlights geometric intuition and typical set exploration.
RELATED TERMS
Markov Chain Monte Carlo Bayesian Inference Stan PyMC Gradient Potential Energy Leapfrog Integrator
COMPETING TERMS
Metropolis-Hastings Slice Sampling No-U-Turn Sampler
RELATIONS
- Broader: Monte Carlo
KNOWLEDGE AREAS
TARGET AUDIENCE
Data Scientist AI Engineer MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Hamiltonian Monte Carlo
- Stan - Probabilistic programming with HMC
- PyMC - PyMC overview including HMC/NUTS
- arXiv - Conceptual introduction to HMC
TOP QUESTIONS
What makes Hamiltonian Monte Carlo efficient?
It uses gradient-informed trajectories to propose distant moves with high acceptance, avoiding slow random walks and improving mixing in high-dimensional posteriors.
Category: Overview
When should I use Hamiltonian Monte Carlo?
Use HMC for differentiable, continuous parameter models where gradients are available and posteriors are complex or high-dimensional.
Category: Requirements
How does HMC compare to Metropolis–Hastings?
HMC leverages gradients to guide proposals, often yielding faster convergence and lower autocorrelation than random-walk Metropolis–Hastings.
Category: Comparison
What are common pitfalls with Hamiltonian Monte Carlo?
Poor step-size or trajectory tuning, bad parameter scaling, and multimodality can impair performance. Reparameterization and diagnostics help.
Category: Mistakes
What is NUTS in relation to Hamiltonian Monte Carlo?
The No-U-Turn Sampler automates trajectory length selection, removing manual tuning while preserving HMC’s efficiency.
Category: Overview
How do I diagnose HMC performance?
Monitor effective sample size, R-hat, energy Bayesian fraction of missing information, divergences, and treedepth to assess sampling quality.
Category: Observability
Does Hamiltonian Monte Carlo handle discrete parameters?
Not directly; HMC requires differentiable densities. Use marginalization or alternative samplers for discrete parameters.
Category: Requirements
Is Hamiltonian Monte Carlo suitable for big data?
It can be costly for massive datasets; subsampling variants exist but can bias results. Exact gradients are preferred.
Category: Risk
Where can I learn Hamiltonian Monte Carlo in practice?
Start with Stan and PyMC tutorials, then study Betancourt’s conceptual notes and advanced diagnostics.
Category: HowTo