DataTerminology.comNEW
Also known as: Evolutionary Strategies
A family of stochastic, gradient-free optimization algorithms inspired by mutation, selection, and recombination.
DESCRIPTION
Evolution Strategies (ES) optimize parameters by sampling populations of candidates, evaluating fitness, and iteratively updating via selection and variation. Methods like CMA-ES adapt covariance to efficiently explore continuous spaces. ES are black-box, making them robust to noisy objectives and non-differentiable functions. They have been applied to reinforcement learning by directly optimizing policies without backpropagation. ES scale with parallel evaluation, but can be sample-inefficient compared to gradient methods. Careful tuning of population size, step-size, and exploration strategies improves performance.
ALTERNATIVE DEFINITIONS
Wikipedia
Evolution strategy (ES) is a subfield of evolutionary computation that deals with optimization of real-valued parameters using mutation, recombination and selection.
Emphasizes real-valued optimization with evolutionary operators.
Scholarpedia
Evolution Strategies (ES) are a class of evolutionary algorithms designed for numerical optimization of continuous parameters.
Highlights continuous-domain optimization and evolutionary algorithm classification.
OpenAI
Evolution Strategies are a family of black-box optimization algorithms that optimize the parameters of a policy by maximizing expected reward.
Frames ES as a scalable alternative to gradient-based RL.
RELATED TERMS
Genetic Algorithm CMA-ES Black-box Optimization Reinforcement Learning Gradient Descent Hyperparameter Tuning Random Search
COMPETING TERMS
Policy Gradient Bayesian Optimization
KNOWLEDGE AREAS
TARGET AUDIENCE
AI Engineer Data Scientist MLOps Engineer Software Developer
EXTERNAL SOURCES
- Wikipedia - Evolution strategy
- OpenAI - ES as scalable RL alternative
- arXiv - CMA-ES tutorial and insights
TOP QUESTIONS
How do Evolution Strategies work?
They sample candidate solutions, evaluate fitness, select the best, and update distributions or parameters using mutation and recombination.
Category: Example
Why use Evolution Strategies over gradient methods?
ES handle noisy, discontinuous objectives and non-differentiable components, and parallelize well for large-scale black-box optimization.
Category: Opinion
What is CMA-ES within Evolution Strategies?
CMA-ES adapts a covariance matrix for sampling, efficiently exploring continuous spaces with advanced step-size and direction control.
Category: Example
What are common mistakes with Evolution Strategies?
Using too small populations or poor step-size control can cause premature convergence or excessive variance.
Category: Mistakes
What is the main risk with Evolution Strategies?
Sample inefficiency can make ES expensive for high-dimensional problems without careful tuning or parallel computation.
Category: Risks
How are Evolution Strategies applied to reinforcement learning?
They directly optimize policy parameters via reward signals, avoiding backpropagation through time or differentiability constraints.
Category: Example
What is currently a hot topic around Evolution Strategies?
Hybrid methods combining ES with gradient learning and distributed ES on large clusters are gaining traction.
Category: Topic
How do Evolution Strategies compare to Bayesian Optimization?
ES scale well in parallel and high dimensions; Bayesian methods excel with sample efficiency and surrogate modeling for expensive objectives.
Category: Alternatives
Where should I start learning Evolution Strategies?
Read Scholarpedia and OpenAI articles, then explore CMA-ES papers and practical libraries.
Category: Example