Back
+20 XP
Concept

Naive Bayes Classifier Implementation

Naive Bayes applies Bayes' Theorem with the assumption of conditional feature independence given the class: $P(C|X) \propto P(C) \prod P(x_i|C)$. Despite the simplification, it delivers fast, highly effective text classification 🛡️.

python
from collections import defaultdict
import math

class NaiveBayes:
    def __init__(self):
        self.class_counts = defaultdict(int)
        self.word_counts = defaultdict(lambda: defaultdict(int))
        self.total_docs = 0

    def train(self, words, label):
        self.class_counts[label] += 1
        self.total_docs += 1
        for w in words:
            self.word_counts[label][w] += 1

    def predict(self, words):
        scores = {}
        v_size = len({w for l in self.word_counts for w in self.word_counts[l]}) or 1
        for label, count in self.class_counts.items():
            score = math.log(count / self.total_docs)
            total = sum(self.word_counts[label].values())
            for w in words:
                w_count = self.word_counts[label].get(w, 0)
                score += math.log((w_count + 1) / (total + v_size))
            scores[label] = score
        return max(scores, key=scores.get)

nb = NaiveBayes()
nb.train(['free', 'lottery', 'cash'], 'spam')
nb.train(['project', 'meeting', 'review'], 'ham')
print(f"Prediction: {nb.predict(['cash', 'lottery'])} 🛡️")

# 53. Naive Bayes Classifier Implementation Naive Bayes applies Bayes' Theorem with the assumption of conditional feature independence given the class: $P(C|X) \propto P(C) \prod P(x_i|C)$. Despite the simplification, it delivers fast, highly effective text classification 🛡️. ```python from collections import defaultdict import math class NaiveBayes: def __init__(self): self.class_counts = defaultdict(int) self.word_counts = defaultdict(lambda: defaultdict(int)) self.total_docs = 0 def train(self, words, label): self.class_counts[label] += 1 self.total_docs += 1 for w in words: self.word_counts[label][w] += 1 def predict(self, words): scores = {} v_size = len({w for l in self.word_counts for w in self.word_counts[l]}) or 1 for label, count in self.class_counts.items(): score = math.log(count / self.total_docs) total = sum(self.word_counts[label].values()) for w in words: w_count = self.word_counts[label].get(w, 0) score += math.log((w_count + 1) / (total + v_size)) scores[label] = score return max(scores, key=scores.get) nb = NaiveBayes() nb.train(['free', 'lottery', 'cash'], 'spam') nb.train(['project', 'meeting', 'review'], 'ham') print(f"Prediction: {nb.predict(['cash', 'lottery'])} 🛡️") ``` Log probabilities are essential: multiplying dozens of tiny probabilities causes floating point underflow to $0.0$. Adding logs keeps computations stable and fast ⚡. --- ### Question Why does the 'Naive' in Naive Bayes describe the model's core assumption? 🧠 - [ ] a. It naively assumes all feature probabilities are mutually independent given the class label 🛡️ - [ ] b. It assumes all input training datasets come from unvalidated user submissions - [ ] c. It skips computing prior class probabilities to minimize arithmetic clock cycles - [ ] d. It is mathematically incapable of handling datasets with more than two categories --- ### Challenge Build a `NaiveBayes` classifier with Laplace smoothing (+1). Train on `(['win', 'money'], 'spam')` and `(['team', 'meeting'], 'ham')`. Predict the label for `['win']` and print it. 🎯 <details> <summary>Solution</summary> ```python from collections import defaultdict import math class NaiveBayes: def __init__(self): self.class_counts = defaultdict(int) self.word_counts = defaultdict(lambda: defaultdict(int)) self.total_docs = 0 def train(self, words, label): self.class_counts[label] += 1 self.total_docs += 1 for w in words: self.word_counts[label][w] += 1 def predict(self, words): scores = {} v_size = len({w for l in self.word_counts for w in self.word_counts[l]}) or 1 for label, count in self.class_counts.items(): score = math.log(count / self.total_docs) total = sum(self.word_counts[label].values()) for w in words: w_count = self.word_counts[label].get(w, 0) score += math.log((w_count + 1) / (total + v_size)) scores[label] = score return max(scores, key=scores.get) nb = NaiveBayes() nb.train(['win', 'money'], 'spam') nb.train(['team', 'meeting'], 'ham') print(nb.predict(['win'])) ``` </details>

Checkpoint

Why does the 'Naive' in Naive Bayes describe the model's core assumption? 🧠

Practice

Build a `NaiveBayes` classifier with Laplace smoothing (+1). Train on `(['win', 'money'], 'spam')` and `(['team', 'meeting'], 'ham')`. Predict the label for `['win']` and print it. 🎯

Terminal
01
Tap code blocks below to assemble
Available Blocks