Classification Algorithm and its types in Machine Learning

This article is just an overview to give you an understanding what is classification Alogirthm and where it stands and what are its various types.So lets start —

As we know, the Supervised Machine Learning algorithm can be broadly classified into Regression and Classification Algorithms.

Lets make it clear — In Regression algorithms, we have predicted the output for continuous values, but to predict the categorical values, we need Classification algorithms.

Supervised vs Unsupervised Learning Algorithms

• In a supervised model, a training dataset in which the targets is also provided with the input data is fed into the model. This type of learning falls under Classification.
• Unsupervised models on the other hand, are fed a dataset that is not labeled and looks for clusters of data points. It can be used to search data for similarities, detect patterns, or identify outliers within a dataset.This type of learning falls under Clustering (Unsupervised models can also be used to find “fraudulent” transactions by looking for anomalies within a dataset.)

So we can say -

The Classification algorithm is a Supervised Learning technique that is used to identify the category of new observations on the basis of training data. Such as, Yes or No, 0 or 1, Spam or Not Spam, cat or dog, etc. Classes can be called as targets/labels or categories.

In classification algorithm, a discrete output function(y) is mapped to input variable(x).

y=f(x), where y=categorical output

Types of classification 🗂

1. Binary Classification- It refers to those classification tasks that have two class labels.

• It is common to model a binary classification task with a model that predicts a Bernoulli probability distribution for each example.
• The class for the normal state is assigned the class label 0 and the class with the abnormal state is assigned the class label 1.
• Popular algorithms that can be used for binary classification include:

* Logistic Regression       
* k-Nearest Neighbors        
* Decision Trees        
* Support Vector Machine        
* Naive Bayes

• Some algorithms are specifically designed for binary classification and do not natively support more than two classes. examples include Logistic Regression and Support Vector Machines.But using a strategy of fitting multiple binary classification models for each class vs. all other classes (called one-vs-rest) or one model for each pair of classes ( called one-vs-one).

2. Multi-Class Classification- Multi-class classification refers to those classification tasks that have more than two class labels. Examples include:Face classification,Plant species classification,Optical character recognition etc.

• Unlike binary classification, multi-class classification does not have the notion of normal and abnormal outcomes. Instead, examples are classified as belonging to one among a range of known classes.
• It is common to model a multi-class classification task with a model that predicts a Multinoulli probability distribution for each example.
• Popular algorithms that can be used for multi-class classification include:

* k-Nearest Neighbors      
* Decision Trees
* Random Forest        
* Gradient Boosting       
* Naive Bayes

3. Multi-Label Classification- Multi-label classification refers to those classification tasks that have two or more class labels, where one or more class labels may be predicted for each example.

• Consider the example of photo classification, where a given photo may have multiple objects in the scene and a model may predict the presence of multiple known objects in the photo, such as “bicycle,” “apple,” “person,” etc.
• This is unlike binary classification and multi-class classification, where a single class label is predicted for each example.
• Classification algorithms used for binary or multi-class classification cannot be used directly for multi-label classification. Specialized versions of standard classification algorithms can be used, so-called multi-label versions of the algorithms, including:

* Multi-label Decision Trees  
* Multi-label Random Forests
* Multi-label Gradient Boosting    
* Another approach is to use a separate classification algorithm to predict the labels for each class

4. Imbalanced Classification- refers to classification tasks where the number of examples in each class is unequally distributed.

• Typically, imbalanced classification tasks are binary classification tasks where the majority of examples in the training dataset belong to the normal class and a minority of examples belong to the abnormal class. Examples include:Fraud detection,Outlier detection,Medical diagnostic tests etc.
• These problems are modelled as binary classification tasks, although may require specialized techniques. Specialized techniques may be used to change the composition of samples in the training dataset by undersampling the majority class or oversampling the majority class. Examples include:

* Random Undersampling
* SMOTE Oversampling

• Specialized modeling algorithms may be used that pay more attention to the minority class when fitting the model on the training dataset, such as cost-sensitive machine learning algorithms.Examples include:

* Cost-sensitive Logistic Regression
* Cost-sensitive Decision Trees
* Cost-sensitive Support Vector Machines

In my future articles , we will look into various classification Algorithm . Thats all for now ..

If you liked the article, show your support by clapping for this article. This article is basically a colab of many articles from medium , analytical vidya , upgrad material etc.

If you are also learning Machine learning like me follow me, for more articles. Lets go on this trip together :)

You can also follow me on Linkedin