DataTerminology.comNEW
Also known as: Logistic Loss
Loss function measuring the divergence between predicted probabilities and true labels, penalizing overconfident wrong predictions.
DESCRIPTION
Log loss (cross-entropy) evaluates probabilistic classifiers by comparing predicted probabilities to actual class labels. It is a proper scoring rule, encouraging truthful probability estimates. In binary settings it reduces to the negative log-likelihood of a Bernoulli model; with softmax it generalizes to multiclass. Minimizing log loss aligns with maximum likelihood under common models. It is sensitive to poorly calibrated or overconfident predictions, making calibration and regularization important.
ALTERNATIVE DEFINITIONS
scikit-learn
Log loss, aka logistic loss or cross-entropy loss, measures the performance of a classification model whose output is a probability value between 0 and 1.
Explicitly frames log loss as measuring probabilistic classifier performance.
Kaggle
Logarithmic loss measures the performance of a classification model where the output is a probability value between 0 and 1.
Introductory explanation focused on probabilistic outputs in practice.
RELATED TERMS
Entropy Softmax KL Divergence Negative Log-Likelihood Calibration Classification Proper Scoring Rule
COMPETING TERMS
RELATIONS
- Equivalent to: Binary Cross-Entropy
KNOWLEDGE AREAS
Mathematics Data Science AI Testing & Validation
TARGET AUDIENCE
Data Scientist MLOps Engineer AI Engineer
EXTERNAL SOURCES
- Wikipedia - Cross-entropy
- Google Developers - ML Crash Course on training and loss
- PyTorch - API for cross-entropy loss in PyTorch
TOP QUESTIONS
Why use Log Loss instead of accuracy?
Log loss rewards well-calibrated probabilities and penalizes overconfidence, providing more nuanced evaluation than binary accuracy counts.
Category: Relevancy
How does Log Loss relate to maximum likelihood?
Minimizing log loss corresponds to maximizing the likelihood under Bernoulli or categorical models, aligning optimization with probabilistic principles.
Category: Concept
What hurts Log Loss the most?
Overconfident wrong predictions cause large penalties. Calibration and regularization help mitigate this sensitivity.
Category: Risks
Is Log Loss appropriate for imbalanced data?
Yes, but calibration and thresholding remain important. Consider complementary metrics like AUROC and PR curves.
Category: Practice
How do you compute multiclass Log Loss?
Use cross-entropy with softmax probabilities summed over classes for each example and averaged across the dataset.
Category: Basics
How does temperature scaling affect Log Loss?
It adjusts probability sharpness to improve calibration, often reducing log loss on validation data.
Category: BestPractices
Is Log Loss a proper scoring rule?
Yes. It incentivizes honest probability estimates, making it suitable for probabilistic forecasting.
Category: Clarification
When might Hinge Loss be preferred over Log Loss?
In maximum-margin classifiers like SVMs focusing on classification boundaries rather than probability calibration.
Category: Alternatives
How should predictions be clipped for Log Loss?
Clip probabilities away from 0 and 1 (e.g., 1e-15 to 1 − 1e-15) to prevent numerical infinities.
Category: Implementation