DataTerminology.comNEW
A reinforcement learning approach that evaluates or improves the same policy used to generate training data.
DESCRIPTION
On-policy methods learn from trajectories generated by the current policy, aligning evaluation and improvement. They include algorithms like SARSA, REINFORCE, and on-policy actor-critic. Stability often improves via trust-region constraints and advantage estimation. On-policy approaches trade sample efficiency for reliable updates and straightforward credit assignment in many environments.
ALTERNATIVE DEFINITIONS
Sutton & Barto (Reinforcement Learning: An Introduction)
On-policy methods evaluate or improve the policy that is used to make decisions.
Canonical textbook distinction between on- and off-policy.
OpenAI Spinning Up
On-policy algorithms are trained using data sampled from the current policy, and their objective typically depends on that policy’s behavior.
Emphasizes data sourcing and objective dependence.
RELATED TERMS
Policy Gradient Actor-Critic Temporal-Difference Learning Exploration Importance Sampling Q-Learning REINFORCE Trust Region Advantage Estimation
COMPETING TERMS
KNOWLEDGE AREAS
TARGET AUDIENCE
AI Engineer Data Scientist MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Reinforcement learning
- arXiv - TRPO: trust region policy optimization
- arXiv - GAE: advantage estimation for on-policy
TOP QUESTIONS
When is On-Policy Learning preferred?
When stability and alignment between behavior and evaluation are critical, such as in safety-sensitive control tasks.
Category: Opinion
What is a common mistake with On-Policy Learning?
Reusing stale trajectories violates assumptions and hinders convergence.
Category: Mistakes
How does On-Policy Learning compare to Off-Policy Learning?
On-policy is less sample-efficient but often more stable; off-policy reuses past data with importance corrections.
Category: Alternatives
What algorithms are On-Policy?
REINFORCE, SARSA, and on-policy actor-critic methods like A2C.
Category: Overview
What are risks with On-Policy exploration?
Insufficient exploration can cause premature convergence; tuning entropy bonuses helps.
Category: Risks
How to improve On-Policy sample efficiency?
Use generalized advantage estimation and trust-region updates to stabilize learning.
Category: Best practices
Can On-Policy be used in continuous control?
Yes; policy gradient and actor-critic methods are widely used in continuous action spaces.
Category: Example
How to monitor On-Policy Learning in production?
Track reward trends, policy entropy, and rollout freshness; retrain frequently to avoid drift.
Category: Management
Where to start learning On-Policy methods?
Read Sutton & Barto, then implement policy gradients with Spinning Up exercises.
Category: Overview