Bayes' Theorem in AI Applications

Bayes' theorem isn't just theoretical—it powers some of the most practical AI systems in use today. This lesson explores how Bayesian reasoning appears in real-world AI applications, from spam filters to medical diagnosis to language models.

Naive Bayes Classification

The Naive Bayes classifier is one of the most successful applications of Bayes' theorem. Despite its simplicity, it remains competitive with much more complex models for many tasks.

How It Works

Given features x₁, x₂, ..., xₙ and classes C:

P(C | x₁, x₂, ..., xₙ) ∝ P(C) × P(x₁ | C) × P(x₂ | C) × ... × P(xₙ | C)

The "naive" assumption is that features are independent given the class.

Spam Detection Example

Features: Words in the email Classes: Spam or Not Spam

Training data teaches us:

• P(Spam) = 0.3 (30% of emails are spam)
• P("free" | Spam) = 0.8
• P("free" | Not Spam) = 0.1
• P("meeting" | Spam) = 0.1
• P("meeting" | Not Spam) = 0.6

For an email containing "free" but not "meeting":

P(Spam | "free") ∝ 0.3 × 0.8 = 0.24
P(Not Spam | "free") ∝ 0.7 × 0.1 = 0.07

Normalizing: P(Spam | "free") = 0.24 / (0.24 + 0.07) = 77.4%

Why Naive Bayes Works

Despite the independence assumption being unrealistic:

1. We only need relative rankings, not exact probabilities
2. It's extremely fast to train and predict
3. Works well with high-dimensional data (many features)
4. Errors from the naive assumption often cancel out

Bayesian Filtering

Email spam filters typically use adaptive Bayesian methods that learn from your behavior:

# Pseudocode for adaptive spam filter
def update_filter(email, is_spam):
    for word in email.words:
        if is_spam:
            word_spam_count[word] += 1
        else:
            word_ham_count[word] += 1

def classify(email):
    log_spam = log(p_spam)  # Prior
    log_ham = log(p_ham)

    for word in email.words:
        log_spam += log(p_word_given_spam[word])
        log_ham += log(p_word_given_ham[word])

    return "spam" if log_spam > log_ham else "ham"

Every time you mark an email as spam or not spam, the filter updates its probabilities.

Medical Diagnosis AI

Bayesian reasoning is crucial in healthcare AI, where:

• False positives cause unnecessary anxiety and procedures
• False negatives miss serious conditions
• Base rates vary widely between conditions

Differential Diagnosis

Given symptoms S₁, S₂, S₃, what diseases are most likely?

P(Disease | Symptoms) ∝ P(Symptoms | Disease) × P(Disease)

A proper Bayesian system considers:

1. Prior probability: How common is this disease?
2. Symptom likelihood: How often do patients with this disease show these symptoms?
3. Patient context: Age, sex, medical history adjust the prior

Example: Chest Pain Diagnosis

P(Heart Attack | Chest Pain, Age=60, Male, Smoker) is calculated by:

• Prior based on demographics
• Likelihood of each symptom given heart attack
• Comparison with alternative diagnoses

Recommendation Systems

Bayesian methods power personalized recommendations:

Collaborative Filtering with Bayesian Models

P(User likes Item | User history, Similar users' preferences)

The system:

1. Maintains a prior for each user-item pair
2. Updates based on user interactions
3. Uses similar users' data as additional evidence

Thompson Sampling for Exploration

When recommending content, we face the exploration-exploitation tradeoff:

• Exploit: Recommend what we think the user will like
• Explore: Try new items to learn more

Thompson Sampling uses Bayesian uncertainty:

1. Sample from the posterior distribution for each item
2. Recommend the item with highest sampled value
3. Uncertain items get explored naturally

Natural Language Processing

Language Models as Probabilistic Systems

Modern language models compute:

P(next_word | previous_words)

While transformers don't use explicit Bayes' theorem, they learn probability distributions that can be interpreted in Bayesian terms:

• Prior: Language patterns learned during pre-training
• Evidence: The context (previous words)
• Posterior: Probability distribution over next words

Bayesian NLP Applications

1. Spelling correction: P(intended_word | typed_word)
2. Named entity recognition: P(entity_type | word, context)
3. Sentiment analysis: P(sentiment | text_features)

Robotics and Sensor Fusion

Robot Localization (SLAM)

A robot asks: "Where am I?"

P(Position | Sensor readings, Map, Previous position)

Using Bayes:

1. Prior: Where the robot was (with uncertainty)
2. Motion model: How movement changes position
3. Sensor model: What sensors should read at each position
4. Update: Combine motion prediction with sensor evidence

Kalman Filters

A Kalman filter is a Bayesian filter for continuous states:

• Tracks position, velocity, etc.
• Fuses multiple noisy sensors
• Used in GPS, self-driving cars, drones

Prediction: x̂ₜ = A·x̂ₜ₋₁ (motion model)
Update: x̂ₜ = x̂ₜ⁻ + K·(zₜ - H·x̂ₜ⁻) (Bayes update)

A/B Testing and Experimentation

Bayesian A/B Testing

Traditional A/B testing: "Is the difference statistically significant?"

Bayesian A/B testing: "What's the probability that A is better than B?"

P(A better than B | Click data) = ?

Benefits:

• Can stop early when confident
• Provides probability statements, not just yes/no
• Handles small samples gracefully
• Naturally incorporates prior knowledge

Multi-Armed Bandits

For online optimization with many options:

For each arm (option):
    Maintain posterior distribution of reward rate
    Sample from posterior
    Pull arm with highest sample
    Update posterior with observed reward

This balances exploration and exploitation optimally.

Anomaly Detection

Bayesian Anomaly Detection

P(Normal | Observation) = P(Observation | Normal) × P(Normal) / P(Observation)

If P(Normal | Observation) is low, the observation is anomalous.

Applications:

• Fraud detection
• Network intrusion detection
• Manufacturing quality control
• Health monitoring

Bayesian Deep Learning

Modern research combines deep learning with Bayesian methods:

Uncertainty Quantification

Standard neural networks give point predictions. Bayesian neural networks give:

• Epistemic uncertainty: Uncertainty about the model (what we don't know)
• Aleatoric uncertainty: Inherent randomness in data

Bayesian Neural Networks

Instead of fixed weights, maintain distributions over weights:

P(Weights | Data) ∝ P(Data | Weights) × P(Weights)

At prediction time, sample weights to get a distribution of outputs.

Applications

• Self-driving cars (knowing when the model is unsure)
• Medical imaging (flagging uncertain diagnoses)
• Active learning (selecting which data points to label)

Summary

Bayes' theorem powers AI applications across domains:

Application	Bayesian Element
Spam filters	Naive Bayes classification
Medical diagnosis	Prior disease rates + symptom likelihoods
Recommendations	Posterior preferences from user history
Robotics	Sensor fusion and localization
A/B testing	Probability that one variant is better
Anomaly detection	P(normal) given observation
Deep learning	Uncertainty in predictions

The key insight: Bayes' theorem provides a principled way to combine prior knowledge with observed evidence—exactly what intelligent systems need to do.

This completes Module 2! Next, we'll explore probability distributions—the mathematical objects that represent uncertainty.