Thursday, January 21, 2016
5 major differences between Naïve Bayes and Logistic Regression.
1. Purpose or what class of machine leaning does it solve?
Both the algorithms can be used for classification of the data. Using these algorithms, you could predict whether a banker can offer a loan to a customer or not or identify given mail is a Spam or ham
2. Algorithm’s Learning mechanism
Naïve Bayes: For the given features (x) and the label y, it estimates a joint probability from the training data. Hence this is a Generative model
Logistic regression: Estimates the probability(y/x) directly from the training data by minimizing error. Hence this is a Discriminative model
3. Model assumptions
Naïve Bayes: Model assumes all the features are conditionally independent .so, if some of the features are dependent on each other (in case of a large feature space), the prediction might be poor.
Logistic regression: It the splits feature space linearly, it works OK even if some of the variables are correlated
4. Model limitations
Naïve Bayes: Works well even with less training data, as the estimates are based on the joint density function
Logistic regression: With the small training data, model estimates may over fit the data
5. Approach to be followed to improve the results
Naïve Bayes: When the training data size is less relative to the features, the information/data on prior probabilities help in improving the results
Logistic regression: When the training data size is less relative to the features, Lasso and Ridge regression will help in improving the results.
Subscribe to:
Posts (Atom)