🔢Target Encoding

NEW

Also known as: Mean Encoding

A technique that replaces categorical values with a statistic of the target variable (often a smoothed mean) to handle high-cardinality features.

DESCRIPTION

Target encoding maps each category to a summary statistic of the target, commonly a mean smoothed toward the global average. It reduces dimensionality versus one-hot encoding, improving generalization with high-cardinality features. However, it risks target leakage if computed on the full training set. Best practice uses cross-validation or out-of-fold schemes and regularization to avoid overfitting. It is widely used in gradient boosting and linear models.

ALTERNATIVE DEFINITIONS

Category Encoders

TargetEncoder replaces a categorical value with the mean of the target variable for that category, optionally using smoothing to reduce overfitting.

Focuses on mean replacement and smoothing to combat overfitting.

Source

Kaggle

Target encoding replaces a categorical value with a number representing the average of the target for that category, often computed in a cross-validation scheme.

Emphasizes cross-validation to prevent leakage.

Source

RELATED TERMS

Categorical Data High Cardinality Smoothing Regularization Cross-Validation Data Leakage K-Fold

COMPETING TERMS

One-Hot Encoding Frequency Encoding Leave-One-Out Encoding

RELATIONS

• Broader: Encoding

KNOWLEDGE AREAS

Data Science AI Testing & Validation

TARGET AUDIENCE

Data Scientist AI Engineer MLOps Engineer

EXTERNAL SOURCES

• Wikipedia - Feature engineering
• Category Encoders - Official TargetEncoder docs
• Feature-engine - Mean (target) encoder docs
• Kaggle - Practical tutorial with CV

TOP QUESTIONS

Why use Target Encoding for high-cardinality features?

It compresses many categories into numeric signals, reducing dimensionality and often improving model performance versus one-hot encoding.

Category: Relevancy

How to prevent leakage with Target Encoding?

Use out-of-fold encodings or nested cross-validation, and apply smoothing and noise to reduce overfitting.

Category: BestPractice

What smoothing strategies are common in Target Encoding?

Weighted averages between category mean and global mean, with weights based on category frequency, are widely used.

Category: Techniques

When is One-Hot Encoding preferable to Target Encoding?

For low-cardinality, non-ordinal categories or when interpretability matters, one-hot encoding can be safer and simpler.

Category: Comparison

Which models benefit most from Target Encoding?

Linear models and gradient boosting frequently gain from target encoding, especially with many rare categories.

Category: Application

What is a common mistake with Target Encoding?

Computing encodings on the full training data without cross-validation causes leakage and inflated validation scores.

Category: Mistakes

How does Target Encoding interact with regularization?

Smoothing acts as a regularizer; adding random noise and using strong model regularization further helps prevent overfit.

Category: Concept

Can Target Encoding be applied to multiclass targets?

Yes, by encoding per class (e.g., one-vs-rest means) or using likelihood-based encodings. Evaluate carefully to avoid leakage.

Category: Techniques

How to deploy Target Encoding in production?

Fit encoders on training data only and store mapping. Apply the same mapping to new data, using fallback values for unseen categories.

Category: Management