DataTerminology.comNEW
Also known as: Target function
Mathematical function that quantifies performance to be minimized or maximized during optimization.
DESCRIPTION
An objective function assigns a scalar score to candidate solutions, guiding optimization toward minima or maxima. In machine learning, it often measures prediction error (e.g., MSE, cross-entropy) or likelihood. Constraints may restrict feasible solutions, forming constrained optimization problems. Convex objectives enable global solutions with efficient algorithms, while nonconvex objectives are harder. Careful choice of the objective ensures alignment with real-world goals and robust model performance.
ALTERNATIVE DEFINITIONS
Google Machine Learning Glossary
A function that you want to minimize or maximize. In machine learning, the objective function is often called a loss function.
Connects the concept directly to ML and loss functions.
Wikipedia
In mathematical optimization, the objective function is the function that is either minimized or maximized.
Concise general definition across optimization contexts.
Stanford CS231n
We define an objective (loss) function that measures the performance of a model, and then minimize it using optimization algorithms such as gradient descent.
Explains objective in the ML training workflow with optimization.
RELATED TERMS
Loss Function Constraints Optimization Algorithm Gradient Descent Regularization Empirical Risk Hyperparameter
COMPETING TERMS
Constraint Function Reward Function
KNOWLEDGE AREAS
Mathematics AI Data Science MLOps
TARGET AUDIENCE
Data Scientist AI Engineer MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Objective function
- Google - ML glossary entry
- Stanford CS231n - Optimization and loss overview
- Boyd & Vandenberghe - Convex optimization foundations
TOP QUESTIONS
How do you choose an objective function?
Select an objective aligned with business or scientific goals. Consider differentiability, robustness to outliers, and evaluation metrics.
Category: Best practices
Is loss function the same as objective function?
In ML, the loss is often the objective. However, objectives may include regularization terms or constraints beyond raw loss.
Category: Misunderstanding
Why are objective functions crucial in ML training?
They formalize what “good performance” means, steering optimization toward desired behavior.
Category: Relevancy
What is a common mistake with objective functions?
Optimizing a proxy inconsistent with real-world goals (e.g., accuracy on imbalanced data) can mislead performance.
Category: Mistakes
How do constraints interact with the objective function?
Constraints limit feasible solutions; the algorithm searches for optima within them.
Category: Requirements
Can objective functions be non-differentiable?
Yes. In such cases, subgradient or gradient-free methods are used.
Category: Requirements
What’s the role of regularization in objectives?
Regularization adds penalties to discourage complexity, improving generalization.
Category: Overview
What is a robust choice for skewed error distributions?
L1-based objectives like MAE are more robust to outliers than MSE.
Category: Best practices
How does the objective relate to evaluation metrics?
Alignment is key; mismatch can yield models optimized for one metric but evaluated on another.
Category: Misunderstanding