DataTerminology.comNEW
A normalizing flow architecture using affine coupling layers for exact likelihood and efficient sampling.
DESCRIPTION
RealNVP (Real-valued Non-Volume Preserving) is a flow-based generative model that applies invertible affine coupling transformations to map data to a simple base distribution. Its Jacobian determinant is tractable, enabling exact log-likelihood computation and stable training. RealNVP supports efficient sampling and inference, unlike autoregressive models that sample sequentially. It advanced density estimation benchmarks and influenced subsequent flows such as Glow. Implementations are available in TensorFlow Probability and PyTorch libraries.
ALTERNATIVE DEFINITIONS
arXiv
We introduce a new class of invertible neural networks, Real NVP, that allows exact computation of the log-likelihood and efficient sampling.
Emphasizes exact likelihood and efficient sampling as core properties.
TensorFlow Probability
RealNVP implements a bijector that uses affine coupling layers to construct invertible transformations with tractable Jacobian determinants.
Explains RealNVP as a bijector component within probabilistic programming.
Lilian Weng
RealNVP builds expressive, invertible mappings by stacking affine coupling layers, enabling exact likelihood via easily computed log-determinants.
Accessible overview stressing affine coupling and log-determinants.
Wikipedia
Normalizing flows are generative models that transform a simple base distribution into a complex one by applying a sequence of invertible mappings with tractable Jacobians.
Places RealNVP within the broader family of normalizing flows.
RELATED TERMS
Normalizing Flow NICE Autoregressive Model Variational Autoencoder GAN Change of Variables Jacobian Determinant Kernel Density Estimation
COMPETING TERMS
Glow Masked Autoregressive Flow
KNOWLEDGE AREAS
TARGET AUDIENCE
AI Engineer Data Scientist MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Normalizing flow
- arXiv - Original RealNVP research paper
- TensorFlow Probability - Official RealNVP bijector docs
- PyTorch - Normalizing flows in PyTorch
TOP QUESTIONS
What makes RealNVP efficient for likelihood computation?
Its affine coupling layers yield triangular Jacobians, making the log-determinant trivial to compute for exact likelihood.
Category: Technique
How does RealNVP compare to autoregressive models?
RealNVP enables parallel sampling, while autoregressive models often sample sequentially, trading off expressiveness and speed.
Category: Comparison
What is a coupling layer in RealNVP?
It splits data into parts; one part is transformed conditioned on the other, preserving invertibility and tractable Jacobians.
Category: Concept
Where is RealNVP used in practice?
It’s used for density estimation, out-of-distribution detection, and as a component in probabilistic programming systems.
Category: Usage
What are common pitfalls when training RealNVP?
Insufficient actnorm or bad initialization can destabilize training; weak base distributions or shallow depth limit expressiveness.
Category: Mistakes
What improved upon RealNVP?
Glow introduced invertible 1×1 convolutions and actnorm, improving image modeling quality and training stability.
Category: Alternatives
How does RealNVP relate to VI and GANs?
RealNVP provides exact likelihood like VI-based models, but uses invertible transforms; unlike GANs, it estimates densities directly.
Category: Background
Is RealNVP suitable for high-resolution images?
Feasible with multi-scale architectures, but memory use can be high; Glow often performs better on large images.
Category: Limitations
What base distributions are typical in RealNVP?
Isotropic Gaussian or logistic distributions are common choices for the latent prior.
Category: Requirements