> For the complete documentation index, see [llms.txt](https://dshub.gitbook.io/ds-hub/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://dshub.gitbook.io/ds-hub/machine-learning/fundamentals/preprocessing/encoding.md).

# Encoding

Encoding is the process of transforming categorical data into a numerical format that machine learning algorithms can interpret. Most algorithms work with numerical inputs, so encoding is essential when working with categorical variables. There are two common methods for encoding:

## **1. Label Encoding**:

* Converts categories into unique integers.  
* Each unique category is assigned an integer in the range \[0, n\_categories - 1] 
  * Example: \["Red", "Green", "Blue"] → \[0, 1, 2]
* Suitable for ordinal data (e.g., "Low", "Medium", "High").
* Simple and memory-efficient
* May introduce unintended ordinal relationships in nominal data, i.e. non-ordinal categorical data, leading to poor performance with algorithms like Linear Regression or K-means.

## **2. One-Hot Encoding**:

* Creates binary columns for each category, indicating presence (`1`) or absence (`0`).
* Converts each unique category into a separate binary column (also known as "dummy variables")
  *  Example: \["Red", "Green", "Blue"] → \[\[1, 0, 0], \[0, 1, 0], \[0, 0, 1]]
* Suitable for nominal data, i.e. no inherit order (e.g., "Red", "Blue", "Green").
* Increases dimensionality but prevents ordinal misinterpretation.
* Prevents introducing ordinal relationships into non-ordinal data.
* Good for algorithms that expect numeric input but don't assume any ordinal relationship (e.g., linear regression, neural networks, clustering).
* Works well with many machine learning models.
* Increases dimensionality significantly when the categorical variable has many unique values.
* Can lead to a sparse dataset.

<details>

<summary>Example Code for Both in Scikit-learn</summary>

```python
from sklearn.preprocessing import LabelEncoder, OneHotEncoder
import pandas as pd

# Sample data
data = {'Color': ['Red', 'Green', 'Blue', 'Red']}
df = pd.DataFrame(data)

# Label Encoding
label_encoder = LabelEncoder()
df['Color_Label'] = label_encoder.fit_transform(df['Color'])

# One-Hot Encoding
onehot_encoder = OneHotEncoder() # add drop = "first" to drop the first categorical variable
color_onehot = onehot_encoder.fit_transform(df[['Color']])
df_onehot = pd.DataFrame(color_onehot, columns=onehot_encoder.get_feature_names_out(['Color']))

# Combine original and one-hot encoded data
df_combined = pd.concat([df, df_onehot], axis=1)

print("Label Encoded Data:")
print(df)

print("\nOne-Hot Encoded Data:")
print(df_combined)


"""
OUTPUT:

Label Encoded Data:
   Color  Color_Label
0    Red            2
1  Green            1
2   Blue            0
3    Red            2

One-Hot Encoded Data:
   Color  Color_Label  Color_Blue  Color_Green  Color_Red
0    Red            2         0.0          0.0        1.0
1  Green            1         0.0          1.0        0.0
2   Blue            0         1.0          0.0        0.0
3    Red            2         0.0          0.0        1.0

"""
```

</details>

## 3. Key Differences

| **Feature**              | **Label Encoding**               | **One-Hot Encoding**                       |
| ------------------------ | -------------------------------- | ------------------------------------------ |
| **Output**               | Single integer column            | Multiple binary columns (one per category) |
| **Type of Data**         | Ordinal or nominal               | Nominal only                               |
| **Ordinal Relationship** | May impose unintended order      | No ordinal relationship implied            |
| **Dimensionality**       | Low (1 column per feature)       | High (one column for each category)        |
| **Use Cases**            | Decision trees, ordinal features | Linear regression, neural networks         |

## 4. Why Encoding Matters

Encoding ensures that categorical features are properly represented numerically, preserving their inherent characteristics while making them compatible with machine learning models. The choice of encoding depends on the type of data and the model requirements.