DataTerminology.comNEW
Also known as: ε-greedy
Select a random action with probability ε; otherwise choose the current best (greedy) action.
DESCRIPTION
ε-greedy is a simple exploration strategy for bandits and RL. With small probability ε, the agent explores randomly; otherwise, it exploits the action with highest estimated value. Annealing ε over time balances early exploration with eventual exploitation. It’s easy to implement and analyze but ignores uncertainty estimates. Alternatives like UCB and Thompson sampling are often more sample-efficient. Deep RL commonly uses ε-greedy with replay buffers.
ALTERNATIVE DEFINITIONS
Wikipedia
In ε-greedy action selection, with probability ε a random action is selected; otherwise the action with the highest estimated value is chosen.
Provides the canonical action-selection rule in one sentence.
Sutton & Barto
The ε-greedy action selection chooses a random action with probability ε; otherwise it chooses the greedy action.
Standard textbook statement of ε-greedy behavior.
RELATED TERMS
Exploration-Exploitation Multi-armed Bandit Q-Learning SARSA Greedy Policy Annealing Schedule Action Selection
COMPETING TERMS
Upper Confidence Bound Thompson Sampling Softmax Exploration
KNOWLEDGE AREAS
TARGET AUDIENCE
Data Scientist AI Engineer MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Multi-armed bandit
- Sutton & Barto - Bandits and ε-greedy
- Lilian Weng - Survey of bandit methods
- OpenAI Spinning Up - Action selection basics
TOP QUESTIONS
When should you use Epsilon-Greedy?
Use ε-greedy when simplicity and ease of implementation are valued. It’s a strong baseline in bandit and tabular RL problems.
Category: Requirements
How should ε be scheduled in Epsilon-Greedy?
Start higher to explore broadly, then anneal toward zero as confidence grows. Adaptive schedules can respond to performance plateaus.
Category: HowTo
What are the drawbacks of Epsilon-Greedy?
It ignores uncertainty and may explore suboptimally. Random exploration can be inefficient compared to UCB or Thompson sampling.
Category: Risks
How does Epsilon-Greedy interact with SARSA and Q-Learning?
It commonly serves as the behavior policy. SARSA learns on-policy under ε-greedy, while Q-learning learns off-policy from ε-greedy behavior.
Category: Example
What are alternatives to Epsilon-Greedy?
Upper Confidence Bound and Thompson sampling offer uncertainty-driven exploration. Softmax (Boltzmann) exploration gives graded action probabilities.
Category: Alternatives
What is a common mistake with Epsilon-Greedy?
Keeping ε too high for too long prevents convergence; too low too early causes premature exploitation. Use annealing and validation.
Category: Mistakes
Is Epsilon-Greedy suitable for continuous action spaces?
Not directly; continuous spaces require different exploration, such as Gaussian noise on actions or parameter space noise.
Category: Misunderstandings
How do you evaluate Epsilon-Greedy performance?
Track regret or cumulative reward across trials. Compare against uncertainty-aware methods to assess efficiency.
Category: Testing
Can Epsilon-Greedy be combined with entropy bonuses?
Yes, in deep RL one can use ε-greedy for action selection and entropy regularization for policy learning. Tune both carefully.
Category: Opinion