19 — Core Math

Bayes' Theorem#

ProbabilityMathematical
◆ The PatternUpdating beliefs with evidence — the foundation of probabilistic ML

Bayes' Theorem tells us how to update a prior belief when we observe new evidence.

P(A|B) = P(B|A) · P(A) / P(B)
P(A|B) = posterior  |  P(B|A) = likelihood  |  P(A) = prior  |  P(B) = evidence
// Medical test — posterior probability after positive result
Prior P(disease) %1.0%
Sensitivity %95%
Posterior—
Base rate fallacy: 1% disease prevalence + 95% accurate test = only ~16% chance you're actually sick after a positive. Low priors dominate!
Pattern bridge: Updating beliefs with evidence is the core of Bayesian statistics. Traders do it intuitively: new data shifts the prior belief — or doesn’t, when confirmation bias blocks the update.
Worked example

A condition that 1% of people have, and a test that catches 99% of cases with 5% false positives. About 5.9% of everyone tests positive, but only 16.7% of positives are sick — the false positives from the healthy 99% outnumber the true ones. A second, independent positive test lifts it to 79.8%.

Where it misleads
  • The answer depends heavily on the prior; the same test means different things in a clinic and in a screening of everyone.
  • Repeated tests are rarely fully independent; the second update assumes they are.
  • People reason much better about this with counts (“10 of 1,000”) than with percentages (Gigerenzer & Hoffrage 1995).
In code
prior = 0.01                          # 1% have the condition
sens, spec = 0.99, 0.95               # P(+ | sick), P(- | healthy)
p_pos = sens * prior + (1 - spec) * (1 - prior)
posterior = sens * prior / p_pos      # P(sick | +)
second = sens * posterior / (sens * posterior + (1 - spec) * (1 - posterior))   # after a second positive test
Python, in your browser — edit the code and run it again

Assumes import numpy as np. Each snippet carries its own example numbers; the comments say which are made up or simulated.

Sources
  • T. Bayes, “An Essay towards Solving a Problem in the Doctrine of Chances”, Philosophical Transactions of the Royal Society 53, 1763
  • G. Gigerenzer & U. Hoffrage, “How to Improve Bayesian Reasoning without Instruction: Frequency Formats”, Psychological Review 102(4), 1995
  • Pattern Recognition and Machine Learning, C. M. Bishop, Springer, 2006

Last reviewed October 2026 · How this site checks its content · Report a mistake

Check yourself

1. A disease affects 1% of people. A test catches 90% of cases and gives a false positive in 9% of healthy people. You test positive. How likely is it that you have the disease?

Show answer

A. Of 1,000 people, 9 sick ones test positive and about 89 healthy ones do too: 9 / 98 ≈ 9%. The low base rate dominates.

2. When does the prior matter most in a Bayesian analysis?

Show answer

C. With much data the likelihood overwhelms any reasonable prior; with little data the prior can decide the answer.

3. What does a naive Bayes classifier assume?

Show answer

C. The assumption is rarely true, yet the classifier often ranks well — its probabilities, though, tend to be overconfident.

Linked from 3
← Previous
KL Divergence
Open in the full reader, with the topic sidebar