Target Encoding (Mean Encoding)

What is Target Encoding?

Target Encoding (also called Mean Encoding) replaces each category in a categorical feature with the mean of the target variable corresponding to that category.

It’s particularly useful in supervised learning when categorical features are related to the target and you want to extract that information.

Why Use Target Encoding?

✅ Great for High-Cardinality Features (e.g., 1,000+ city names)

✅ Captures Relationship Between Feature and Target

✅ Keeps Model Input Compact (unlike One-Hot Encoding)

🚫 Caution:

• Can cause data leakage if not handled properly (e.g., target values influencing training).
• Must be used with cross-validation or out-of-fold encoding.

How Target Encoding Works

1. Group the dataset by the categorical feature.
2. Calculate the mean target value for each category.
3. Replace the category with its corresponding mean value.

Example

Let’s say we’re predicting sales and have a City column:

Input Dataset

Step 1: Compute Mean Target per Category

• London → (200 + 220) / 2 = 210
• Paris → (250 + 270) / 2 = 260
• New York → 300

Step 2: Replace Category with Mean Target

Implementing Target Encoding in Python

✅ Using category_encoders library

Handling Data Leakage

To avoid data leakage, especially during model validation:

• Use out-of-fold encoding for training data
• Apply encoding separately on validation/test sets

✅ Using Cross-Validation Encoding

Comparison with Other Encoders

When to Use Target Encoding

✅ When categorical feature is strongly related to the target

✅ When dealing with high-cardinality columns

✅ In regression or classification tasks

🚫 Avoid when:

• You're working with test data that has unseen categories
• You're not using proper validation techniques

Conclusion

Target Encoding is a powerful technique for handling categorical features with many unique values. It captures the relationship between categories and the target, but must be used carefully to prevent leakage. Always validate properly. 🚀

• What It Does: Replaces categories with the mean of the target variable for each category.
• Best For: Commonly used for high-cardinality categorical features.
• Caution: Risk of data leakage; apply carefully with validation techniques.
• Example:
• For a "city" feature with values ["London", "Paris", "New York"], calculate the mean target value for each city.

mean_encoding = df.groupby('city')['target'].mean().to_dict()
df['city_target_mean'] = df['city'].map(mean_encoding)
print(df[['city', 'city_target_mean']])