DataTerminology.comNEW
Also known as: Multinoulli Distribution
A discrete probability distribution over a finite set of categories, parameterized by class probabilities that sum to one.
DESCRIPTION
The categorical distribution models a single draw from K discrete classes, assigning each class a probability. It generalizes the Bernoulli distribution (K=2) and is fundamental to classification models. Softmax outputs in neural networks parameterize categorical likelihoods, often optimized via cross-entropy. Bayesian models commonly use the Dirichlet prior for categorical probabilities. It underlies sampling, label noise modeling, and probabilistic programming primitives.
ALTERNATIVE DEFINITIONS
Wikipedia
In probability theory and statistics, the categorical distribution is a discrete probability distribution that describes the possible results of a random variable that can take on one of K possible categories, with the probability of each category specified.
Highlights K categories with specified probabilities; general framing across probability and statistics.
PyTorch
Creates a categorical distribution parameterized by either probabilities or logits over K categories.
Emphasizes parameterization via probabilities or logits in software implementation.
TensorFlow Probability
Categorical distribution over class indices.
Concise implementation-oriented definition focusing on discrete class indices.
StatLect
The categorical distribution is the generalization of the Bernoulli distribution to variables with more than two categories.
Connects to Bernoulli as a special case (K=2) and extends intuition.
RELATED TERMS
Softmax One-Hot Encoding Dirichlet Distribution Log Loss Probability Mass Function
COMPETING TERMS
Bernoulli Distribution Multinomial Distribution
RELATIONS
- Broader: Probability Mass Function
KNOWLEDGE AREAS
TARGET AUDIENCE
Data Scientist AI Engineer MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Categorical distribution
- PyTorch - API for categorical distribution
- TensorFlow Probability - TFP categorical distribution docs
- StatLect - Theory and properties overview
TOP QUESTIONS
What is the Categorical Distribution used for?
It models a single trial with K possible classes, each with a probability. It’s the likelihood behind multiclass classification and many probabilistic models.
Category: Use Cases
How does the Categorical Distribution relate to Bernoulli?
Bernoulli is a special case with K=2 categories. The categorical generalizes this to K>2 classes.
Category: Concept
What are common parameterizations of the Categorical Distribution?
It can be parameterized by probabilities that sum to one, or by logits that softmax to probabilities.
Category: Definition
How is the Categorical Distribution used in neural networks?
Softmax outputs define categorical probabilities, and cross-entropy measures fit to one-hot labels or label distributions.
Category: Application
What is a suitable prior for the Categorical Distribution?
The Dirichlet distribution is a conjugate prior for the categorical probabilities.
Category: Theory
What’s a common mistake when using the Categorical Distribution?
Forgetting probabilities must sum to one or misaligning label indices can invalidate likelihoods.
Category: Mistakes
How do I sample from a Categorical Distribution efficiently?
Use cumulative sums and inversion sampling or library routines that accept probabilities or logits.
Category: Practical
What is the difference between Categorical and Multinomial distributions?
Categorical is a single draw; multinomial is multiple draws with counts across classes.
Category: Differences
When is Categorical Distribution inappropriate?
When outcomes are ordinal or continuous; use ordinal models or continuous distributions instead.
Category: Risks