what is nave bayes algorithm?

the naive bayes algorithm is a classification algorithm based on bayes' theorem. the algorithm assumes that the features are independent of each other, which is why it is called "naive." it calculates the probability of a sample belonging to a particular class based on the probabilities of its features. for example, a phone may be considered as smart if it has touch-screen, internet facility, good camera, etc. even if all these features are dependent on each other, but all these features independently contribute to the probability of that the phone is a smart phone.

in bayesian classification, the main interest is to find the posterior probabilities i.e. the probability of a label given some observed features, p(l | features). with the help of bayes theorem, we can express this in quantitative form as follows −

$$p\left ( l| features\right )=\frac{p\left ( l \right )p\left (features| l\right )}{p\left (features\right )}$$

here,

$p\left ( l| features\right )$ is the posterior probability of class.
$p\left ( l \right )$ is the prior probability of class.
$p\left (features| l\right )$ is the likelihood which is the probability of predictor given class.
$p\left (features\right )$ is the prior probability of predictor.

in the naive bayes algorithm, we use bayes' theorem to calculate the probability of a sample belonging to a particular class. we calculate the probability of each feature of the sample given the class and multiply them to get the likelihood of the sample belonging to the class. we then multiply the likelihood with the prior probability of the class to get the posterior probability of the sample belonging to the class. we repeat this process for each class and choose the class with the highest probability as the class of the sample.

types of naive bayes algorithm

there are many types of naive bayes algorithm. here we discuss the following three types −

gaussian nave bayes

gaussian nave bayes is the simplest nave bayes classifier having the assumption that the data from each label is drawn from a simple gaussian distribution. it is used when the features are continuous variables that follow a normal distribution.

multinomial nave bayes

another useful nave bayes classifier is multinomial nave bayes in which the features are assumed to be drawn from a simple multinomial distribution. such kind of nave bayes are most appropriate for the features that represents discrete counts. it is commonly used in text classification tasks where the features are the frequency of words in a document.

bernoulli nave bayes

another important model is bernoulli nave bayes in which features are assumed to be binary (0s and 1s). text classification with 'bag of words' model can be an application of bernoulli nave bayes.

implementation of nave bayes algorithm in python

depending on our data set, we can choose any of the nave bayes model explained above. here, we are implementing gaussian nave bayes model in python −

we will start with required imports as follows −

import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns; sns.set()

now, by using make_blobs() function of scikit learn, we can generate blobs of points with gaussian distribution as follows −

from sklearn.datasets import make_blobs
x, y = make_blobs(300, 2, centers=2, random_state=2, cluster_std=1.5)
plt.scatter(x[:, 0], x[:, 1], c=y, s=50, cmap='summer');

next, for using gaussiannb model, we need to import and make its object as follows −

from sklearn.naive_bayes import gaussiannb
model_gnb = gaussiannb()
model_gnb.fit(x, y);

now, we have to do prediction. it can be done after generating some new data as follows −

rng = np.random.randomstate(0)
xnew = [-6, -14] + [14, 18] * rng.rand(2000, 2)
ynew = model_gnb.predict(xnew)

next, we are plotting new data to find its boundaries −

plt.scatter(x[:, 0], x[:, 1], c=y, s=50, cmap='summer')
lim = plt.axis()
plt.scatter(xnew[:, 0], xnew[:, 1], c=ynew, s=20, cmap='summer', alpha=0.1)
plt.axis(lim);

now, with the help of following line of codes, we can find the posterior probabilities of first and second label −

yprob = model_gnb.predict_proba(xnew)
yprob[-10:].round(3)

output

array([[0.998, 0.002],
   [1.   , 0.   ],
   [0.987, 0.013],
   [1.   , 0.   ],
   [1.   , 0.   ],
   [1.   , 0.   ],
   [1.   , 0.   ],
   [1.   , 0.   ],
   [0.   , 1.   ],
   [0.986, 0.014]]
)

pros & cons of nave bayes classification

let's discuss some of the advantages and limitations of naive bayes classification algorithm.

pros

the followings are some pros of using nave bayes classifiers −

nave bayes classification is easy to implement and fast.
it will converge faster than discriminative models like logistic regression.
it requires less training data.
it is highly scalable in nature, or they scale linearly with the number of predictors and data points.
it can make probabilistic predictions and can handle continuous as well as discrete data.
nave bayes classification algorithm can be used for binary as well as multi-class classification problems both.

cons

the followings are some cons of using nave bayes classifiers −

one of the most important cons of nave bayes classification is its strong feature independence because in real life it is almost impossible to have a set of features which are completely independent of each other.
another issue with nave bayes classification is its 'zero frequency' which means that if a categorial variable has a category but not being observed in training data set, then nave bayes model will assign a zero probability to it and it will be unable to make a prediction.

applications of nave bayes classification

the following are some common applications of nave bayes classification −

real-time prediction − due to its ease of implementation and fast computation, it can be used to do prediction in real-time.

multi-class prediction − nave bayes classification algorithm can be used to predict posterior probability of multiple classes of target variable.

text classification − due to the feature of multi-class prediction, nave bayes classification algorithms are well suited for text classification. that is why it is also used to solve problems like spam-filtering and sentiment analysis.

recommendation system − along with the algorithms like collaborative filtering, nave bayes makes a recommendation system which can be used to filter unseen information and to predict weather a user would like the given resource or not.

Save or share this article