DataTerminology.comNEW
A likelihood-based generative approach using invertible transformations to map a simple base distribution to complex data distributions exactly.
DESCRIPTION
Normalizing flows compose bijective transformations with tractable Jacobians to model complex densities with exact log-likelihoods. They enable sampling and inference by inverting mappings between latent and data spaces. Architectures like RealNVP and Glow use coupling layers to keep determinants efficient. Flows are used in density estimation, variational inference, and as components in hybrid generative systems. Trade-offs include expressivity versus invertibility constraints.
ALTERNATIVE DEFINITIONS
Wikipedia
In machine learning, a normalizing flow is a generative model that constructs complex probability distributions by transforming a simple base distribution using a sequence of invertible mappings.
Stresses invertible mappings from base to complex distributions.
arXiv
Normalizing flows are likelihood-based generative models that transform a simple distribution into a more complex one by composing invertible functions with tractable Jacobians.
Highlights exact likelihood and tractable Jacobians in the definition.
RELATED TERMS
Change of Variables Invertible Neural Network Likelihood-Based Model RealNVP Glow Autoregressive Model Energy-Based Model
COMPETING TERMS
Variational Autoencoder GAN Diffusion Models
RELATIONS
- Broader: Flow
KNOWLEDGE AREAS
TARGET AUDIENCE
AI Engineer Data Scientist MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Normalizing flow
- Dinh et al. - RealNVP coupling layers
- Kingma and Dhariwal - Glow generative flow
- Wikipedia - Overview and references
TOP QUESTIONS
Why use Normalizing Flows over VAEs or GANs?
Flows provide exact likelihoods and invertible sampling, aiding evaluation and uncertainty estimation. However, architectural constraints can limit expressivity.
Category: Comparison
How do coupling layers enable efficient Flows?
They apply invertible transformations to partitions of the input, yielding triangular Jacobians with cheap determinants.
Category: Concepts
What are common use cases for Normalizing Flows?
Density estimation, variational inference, outlier detection, and as priors in hybrid generative models.
Category: Examples
What are pitfalls when training Flows?
Poor initialization, unstable scaling, and insufficient depth can limit expressivity. Data preprocessing is crucial.
Category: Mistakes
How do Flows compare to Diffusion Models?
Flows give exact likelihoods and fast sampling but can be less expressive; diffusion often yields higher fidelity but slower sampling.
Category: Comparison
What metrics evaluate Flows?
Negative log-likelihood on test data and sample quality metrics like FID for images.
Category: Concepts
Can Normalizing Flows model discrete data?
With dequantization or relaxed representations, flows can model discrete data like images or text tokens.
Category: Concepts
Where should I start learning about Normalizing Flows?
Begin with RealNVP and Glow papers, then read survey articles covering design choices and applications.
Category: Learning
What is the biggest misconception about Flows?
That invertibility guarantees expressivity; constraints often necessitate deep stacks or hybrid models for complex data.
Category: Misunderstanding