> For the complete documentation index, see [llms.txt](https://dshub.gitbook.io/ds-hub/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://dshub.gitbook.io/ds-hub/machine-learning/fundamentals/supervised-learning.md).

# Supervised Learning

## 1. Description

Supervised learning is a type of machine learning in which an algorithm learns a mapping or relationship between input data and corresponding output labels from a labeled training dataset. In other words, it involves training a model to make predictions or classifications based on input features while having access to the correct answers during training. The goal of supervised learning is to learn a mapping function that can generalize to make accurate predictions on new, unseen data.

<figure><img src="https://483934582-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F332OmkaCCBc9TZFZXfnO%2Fuploads%2FdeeWGeGUU7sEtRVU0Btv%2Fsl.png?alt=media&amp;token=dbf4f15d-b7a6-4d55-8c4a-d9016b835f6d" alt=""><figcaption></figcaption></figure>

There are two primary types of supervised learning:

1. **Regression:**

   * Regression involves predicting a continuous numerical value or quantity based on input features. In regression, the output is a real number rather than a discrete category. The model learns to approximate the relationship between input variables and a target numerical value.
   * Examples of regression tasks include:
     * Predicting housing prices based on features like square footage, number of bedrooms, and location.
     * Forecasting stock prices based on historical market data.
     * Estimating the age of a person based on demographic information.

   Regression algorithms include linear regression, polynomial regression, support vector regression, and various flavors of regression in machine learning libraries.
2. **Classification:**

   * In classification, the goal is to categorize input data into discrete classes or categories. Each data point is associated with a specific class label. The model's objective is to learn the decision boundaries that separate different classes in the feature space.
   * Examples of classification tasks include:
     * Email spam detection (classifying emails as spam or not spam).
     * Image classification (e.g., classifying images of animals into different species).
     * Sentiment analysis (classifying text as positive, negative, or neutral).

   Common algorithms for classification include logistic regression, decision trees, random forests, support vector machines, and deep learning techniques like neural networks.

## **2. Algorithms**

Some major supervised learning algorithms into based on regression and classification algorithms.

### **2.1. Regression Algorithms:**

1. **Linear Regression**
   1. Ridge
   2. Lasso
2. **Multiple Linear Regression**
3. **Polynomial Regression**

### **2.2. Classification Algorithms:**

1. **K-Nearest Neighbors** (K-NN) (also can be  used for regression)
2. **Logistic Regression**
3. **Decision Trees**
   * Random Forests (also can be used for regression)
   * Gradient Boosting Machines (GBM) (also can be used for regression)
4. **Support Vector Machines** (SVM)
5. **Naive Bayes**
   * Gaussian
   * Bernoulli
   * Multinomial
6. **Neural Networks** (Deep Learning)
   * Convolutional Neural Networks (CNNs)
   * Recurrent Neural Networks (RNNs)
7. **Linear Discriminant Analysis** (LDA)

## Steps

<figure><img src="https://483934582-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F332OmkaCCBc9TZFZXfnO%2Fuploads%2Fwrj17FeWi648C7C1iH1G%2Fsl_types.png?alt=media&amp;token=4296c9eb-844b-4e51-b87c-ab87aa20837c" alt=""><figcaption></figcaption></figure>

The key steps in supervised learning include:

1. **Data Collection:** Gathering a labeled dataset consisting of input features and corresponding output labels.
2. **Data Preprocessing:** Preparing and cleaning the data, which may involve tasks like feature scaling, handling missing values, and encoding categorical variables.

{% hint style="info" %}
We will be using one of the most popular machine learning packages, called [Scikit-learn](https://scikit-learn.org/stable/). It is known for its easy to use interface and its range of functions and methods for building and training machine learning models.

The scikit-learn requires:

Data:

1. No missing values

2. All numeric (1 or 0 instead of Yes/No or True/False), i.e. NO categorical data

3. Features must be formatted as a 2D array:&#x20;
   * either Pandas DataFrame or Numpy’s 2d array&#x20;

4. Target should be a 1d array:
   * y = data\['target'].values
   * y = np.ravel(y) -> to convert to 1d
     {% endhint %}

5. **Model Selection:** Choosing an appropriate supervised learning algorithm based on the nature of the problem and the dataset.

6. **Training:** Using the labeled training data to train the selected model. During training, the model adjusts its parameters to minimize the prediction error.

7. **Evaluation:** Assessing the model's performance using evaluation metrics such as accuracy, precision, recall, mean squared error, or others, depending on the task.

8. **Testing:** Testing the trained model on unseen data (the testing dataset) to measure its ability to generalize to new examples.

9. **Deployment:** Deploying the trained model in real-world applications to make predictions or classifications.

Supervised learning is widely used in various domains, including natural language processing, computer vision, healthcare, finance, and more, where there is a need to make predictions or decisions based on historical data and labeled examples.