DataTerminology.comNEW
Also known as: Posterior Sampling
A Bayesian bandit method that selects actions by sampling from posterior reward distributions to balance exploration and exploitation.
DESCRIPTION
Thompson Sampling maintains a posterior belief over each arm’s expected reward and selects actions by sampling from these distributions. This naturally balances exploration and exploitation according to uncertainty. It enjoys strong empirical performance and theoretical guarantees under various reward models. The method is widely applied in online experimentation, recommendations, and advertising. Implementation hinges on tractable posteriors or approximations such as conjugate priors or Bayesian updates.
ALTERNATIVE DEFINITIONS
Wikipedia
Thompson sampling, also known as posterior sampling, is a heuristic for choosing actions that maximizes expected reward by sampling from the posterior distribution of each action’s mean reward.
Explains the sampling-based selection from posterior beliefs.
Russo & Van Roy
Thompson sampling selects actions according to their probability of being optimal, as determined by a Bayesian posterior distribution.
Highlights the probability-of-optimality decision rule under Bayesian posterior.
RELATED TERMS
Bayesian Inference Multi-armed Bandit Posterior Exploration-Exploitation Regret Minimization
COMPETING TERMS
Upper Confidence Bound Epsilon-Greedy
RELATIONS
- Broader: Sampling
KNOWLEDGE AREAS
AI Data Science Testing & Validation
TARGET AUDIENCE
Data Scientist AI Engineer AI Product Manager
EXTERNAL SOURCES
- Wikipedia - Thompson sampling
- Russo & Van Roy - Overview and theory of Thompson Sampling
- Lilian Weng - Practical bandit algorithms tutorial
- Google Data Science - Educational bandit overview
TOP QUESTIONS
How does Thompson Sampling balance exploration and exploitation?
Sampling from posterior distributions naturally explores uncertain arms while exploiting high-probability optimal arms, without explicit bonuses.
Category: Concept
What priors are used in Thompson Sampling?
Conjugate priors like Beta-Bernoulli or Gaussian-Gaussian are common, enabling simple Bayesian updates and tractable posteriors.
Category: Requirements
How does Thompson Sampling compare to UCB?
UCB is optimistic and frequentist; Thompson Sampling is Bayesian and probabilistic. Thompson often performs well empirically with natural uncertainty handling.
Category: Comparison
Is Thompson Sampling suitable for nonstationary environments?
Yes, with modifications like discounting, sliding windows, or dynamic priors to track changing reward distributions.
Category: Application
What are common mistakes implementing Thompson Sampling?
Mismatched likelihood-prior pairs, numerical instability in updates, or ignoring delayed feedback can degrade performance.
Category: Mistakes
What is currently hot around Thompson Sampling?
Extensions for contextual bandits, deep Bayesian approaches, and applications in large-scale recommendation systems.
Category: Topic
Can Thompson Sampling handle continuous rewards?
Yes, with appropriate likelihood models (e.g., Gaussian) and priors that yield tractable posterior sampling.
Category: Scope
How does posterior uncertainty drive decisions in Thompson Sampling?
Higher uncertainty increases the chance an arm is sampled and tried, aligning exploration with epistemic uncertainty.
Category: Mechanics
How do you validate Thompson Sampling policies?
Use offline replay simulation, counterfactual evaluation, or controlled online experiments with guardrails.
Category: Testing