DataTerminology.comNEW
Also known as: Class Weighting, Cost-Sensitive Loss
Loss functions adjusted with class or sample weights to penalize errors unevenly.
DESCRIPTION
Weighted losses assign higher penalties to underrepresented or critical classes to counter imbalance and reflect real-world costs. Common implementations include class-weighted cross-entropy and focal loss. Weights can be defined manually, computed from class frequencies, or tuned via validation. They improve recall for minority classes and allow cost-sensitive optimization. Careful selection avoids instability or overemphasis on rare noise.
ALTERNATIVE DEFINITIONS
PyTorch
weight (Tensor, optional) – a manual rescaling weight given to each class.
Defines per-class weighting within cross-entropy to rescale class contributions to loss.
TensorFlow
class_weight: Optional dictionary mapping class indices (integers) to a weight (float) to apply to the model’s loss for the samples of the class.
Clarifies class-level weighting applied during model.fit to adjust loss contribution.
RELATED TERMS
Focal Loss Class Imbalance Imbalanced Data Cross-Entropy Loss Weighting Cost-Sensitive Learning Precision-Recall Curve Threshold Tuning
COMPETING TERMS
KNOWLEDGE AREAS
AI Data Science MLOps Testing & Validation
TARGET AUDIENCE
AI Engineer Data Scientist MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Loss function
- PyTorch - Class-weighted cross-entropy docs
- TensorFlow - Keras class_weight parameter
- scikit-learn - Compute class weights utility
TOP QUESTIONS
What is Weighted Loss and why use Weighted Loss?
Weighted loss scales errors by class or sample importance. It combats class imbalance and encodes cost-sensitive decisions.
Category: Overview
How do I choose class weights for Weighted Loss?
Use inverse class frequency, domain costs, or hyperparameter tuning. Validate to avoid instability.
Category: HowTo
What is a common mistake with Weighted Loss?
Overweighting rare noisy classes can harm generalization. Start with moderate weights and monitor metrics.
Category: Mistakes
What risks arise when using Weighted Loss?
Training instability and overfitting minority noise are risks. Adjust learning rates or regularization accordingly.
Category: Risks
What are alternatives to Weighted Loss?
Oversampling, undersampling, SMOTE, and focal loss are alternatives or complements.
Category: Alternatives
How does Weighted Loss affect metrics?
It typically improves minority recall and F1 while keeping precision in check when tuned properly.
Category: Opinion
Can Weighted Loss be combined with resampling?
Yes; combining class weights with modest resampling often yields balanced improvements.
Category: BestPractice
How to implement Weighted Loss in common frameworks?
Use class_weight in Keras, weight tensors in PyTorch, or class_weight utilities in scikit-learn.
Category: Requirements
What is currently a hot topic around Weighted Loss?
Focal losses, label smoothing with weights, and uncertainty-aware weighting are active areas.
Category: Topic