Naive Bayes Classifier Implementation
Naive Bayes applies Bayes' Theorem with the assumption of conditional feature independence given the class: $P(C|X) \propto P(C) \prod P(x_i|C)$. Despite the simplification, it delivers fast, highly effective text classification 🛡️.
from collections import defaultdict
import math
class NaiveBayes:
def __init__(self):
self.class_counts = defaultdict(int)
self.word_counts = defaultdict(lambda: defaultdict(int))
self.total_docs = 0
def train(self, words, label):
self.class_counts[label] += 1
self.total_docs += 1
for w in words:
self.word_counts[label][w] += 1
def predict(self, words):
scores = {}
v_size = len({w for l in self.word_counts for w in self.word_counts[l]}) or 1
for label, count in self.class_counts.items():
score = math.log(count / self.total_docs)
total = sum(self.word_counts[label].values())
for w in words:
w_count = self.word_counts[label].get(w, 0)
score += math.log((w_count + 1) / (total + v_size))
scores[label] = score
return max(scores, key=scores.get)
nb = NaiveBayes()
nb.train(['free', 'lottery', 'cash'], 'spam')
nb.train(['project', 'meeting', 'review'], 'ham')
print(f"Prediction: {nb.predict(['cash', 'lottery'])} 🛡️")
# 53. Naive Bayes Classifier Implementation Naive Bayes applies Bayes' Theorem with the assumption of conditional feature independence given the class: $P(C|X) \propto P(C) \prod P(x_i|C)$. Despite the simplification, it delivers fast, highly effective text classification 🛡️. ```python from collections import defaultdict import math class NaiveBayes: def __init__(self): self.class_counts = defaultdict(int) self.word_counts = defaultdict(lambda: defaultdict(int)) self.total_docs = 0 def train(self, words, label): self.class_counts[label] += 1 self.total_docs += 1 for w in words: self.word_counts[label][w] += 1 def predict(self, words): scores = {} v_size = len({w for l in self.word_counts for w in self.word_counts[l]}) or 1 for label, count in self.class_counts.items(): score = math.log(count / self.total_docs) total = sum(self.word_counts[label].values()) for w in words: w_count = self.word_counts[label].get(w, 0) score += math.log((w_count + 1) / (total + v_size)) scores[label] = score return max(scores, key=scores.get) nb = NaiveBayes() nb.train(['free', 'lottery', 'cash'], 'spam') nb.train(['project', 'meeting', 'review'], 'ham') print(f"Prediction: {nb.predict(['cash', 'lottery'])} 🛡️") ``` Log probabilities are essential: multiplying dozens of tiny probabilities causes floating point underflow to $0.0$. Adding logs keeps computations stable and fast ⚡. --- ### Question Why does the 'Naive' in Naive Bayes describe the model's core assumption? 🧠 - [ ] a. It naively assumes all feature probabilities are mutually independent given the class label 🛡️ - [ ] b. It assumes all input training datasets come from unvalidated user submissions - [ ] c. It skips computing prior class probabilities to minimize arithmetic clock cycles - [ ] d. It is mathematically incapable of handling datasets with more than two categories --- ### Challenge Build a `NaiveBayes` classifier with Laplace smoothing (+1). Train on `(['win', 'money'], 'spam')` and `(['team', 'meeting'], 'ham')`. Predict the label for `['win']` and print it. 🎯 <details> <summary>Solution</summary> ```python from collections import defaultdict import math class NaiveBayes: def __init__(self): self.class_counts = defaultdict(int) self.word_counts = defaultdict(lambda: defaultdict(int)) self.total_docs = 0 def train(self, words, label): self.class_counts[label] += 1 self.total_docs += 1 for w in words: self.word_counts[label][w] += 1 def predict(self, words): scores = {} v_size = len({w for l in self.word_counts for w in self.word_counts[l]}) or 1 for label, count in self.class_counts.items(): score = math.log(count / self.total_docs) total = sum(self.word_counts[label].values()) for w in words: w_count = self.word_counts[label].get(w, 0) score += math.log((w_count + 1) / (total + v_size)) scores[label] = score return max(scores, key=scores.get) nb = NaiveBayes() nb.train(['win', 'money'], 'spam') nb.train(['team', 'meeting'], 'ham') print(nb.predict(['win'])) ``` </details>