Naive Bayes from Scratch

What is Naive Bayes?

In machine learning, the Naive Bayes is a classification algorithm based on the concept of Bayes Theorem. Bayes theorem is one of the fundamental theorems in probability. We will be learning all these concepts in the course quite thoroughly

How are Naive Bayes and probability related?

The Bayes Theorem forms the backbone of the Naive Bayes algorithm. We use conditional probability to classify the data - thus, the Naive Bayes algorithm basically gives us the probability of a record being in a particular class, given the values of the features. This is nothing but conditional probability!

What are the assumptions we need to keep in mind for using the Naive Bayes algorithm?

The most important assumption that Naive Bayes makes is that all the features are independent of each other. You might also need to convert the continuous variables into discrete variables. 

What are the different types of Naive Bayes algorithms?

There are 3 main types of Naive Bayes algorithms:

• Gaussian Naive Bayes
• Multinomial Naive Bayes
• Bernoulli Naive Bayes

We will be covering all these techniques comprehensively and with Python code in this course.

Can Naive Bayes be used only for binary classification?

Not at all - in fact, the Naive Bayes classifier is one of the most popular algorithms for multiclass classification. Depending on the data we are dealing with, we can use any of the 3 types of the Naive Bayes algorithms

How does Naive Bayes hold up against other classification algorithms?

Though classification algorithms like Logistic Regression, random forest, etc are quite popular, Naive Bayes holds its own among them. It is faster than Random forest, since it can adapt to changing data pretty quickly. If the assumptions of Naive Bayes hold true, then it is much faster than logistic regression as well.

What are the main advantages of Naive Bayes?

The main advantages are:

• It is fast, and easy to understand
• It is not prone to overfitting
• It does not need much training data

What are the main advantages of Naive Bayes?

Like all other algorithms, Naive Bayes has its own set of cons as well:

• It does not work that well when the number of features is very high
• The assumption that input features are independent of each other may not always hold true
• You might lose important information while discretising the continuous variables.

What are the applications of the Naive Bayes algorithm?

Though the algorithm is relatively simple to understand, you cannot underestimate the power of the Naive Bayes classifier! It is fast, intuitive, and is used for text classification tasks. Since it can be used for multiclass classification as well, it is considered a very versatile and flexible classifier. A majority of research papers on text classification start off by using the Naive Bayes classifier as the baseline model.