Conditional probability and Bayes

Conditional probability

Knowing that one event has happened changes the chances of another. The conditional probability of \(A\) given \(B\), written \(P(A \mid B)\), is the probability of \(A\) once we know that \(B\) has happened. Knowing \(B\) shrinks the sample space to \(B\), so we count the share of \(B\) that is also in \(A\):

Among the clinic's 150 patients from chapter 1, 54 feel fatigue and 27 of them also have a fever, so \(P(\text{fever} \mid \text{fatigue}) = \tfrac{27}{54} = 0.5\). The other direction is a different question with a different answer: of the 81 patients with a fever, 27 feel fatigue, so \(P(\text{fatigue} \mid \text{fever}) = \tfrac{27}{81} = 0.3333\).

The multiplication rule

Turning the definition around gives the probability that both events happen: the chance of the first, times the chance of the second given the first.

Check it on the patients: \(P(\text{fatigue}) \times P(\text{fever} \mid \text{fatigue}) = 0.36 \times 0.5 = 0.18\), the 27 patients with both symptoms.

Independent events

Two events are independent when knowing that one has happened does not change the chance of the other: \(P(A \mid B) = P(A)\). The multiplication rule then becomes a plain product:

Two flips of a coin are independent: \(P(\text{two heads}) = 0.5 \times 0.5 = 0.25\). The symptoms are not: a fever has probability 0.54 overall but 0.5 among patients with fatigue, and \(0.54 \times 0.36 = 0.1944\), not the 0.18 the clinic sees.

Independent is not the same as mutually exclusive. Mutually exclusive events cannot happen together, so knowing that one happened tells you the other did not: they are as far from independent as events can be. A 1 and a 6 on the same roll are mutually exclusive, and \(P(\text{6} \mid \text{1}) = 0\), not \(\tfrac{1}{6}\).

Tree diagrams and total probability

A tree diagram draws a two-stage experiment: the first branches split on one event, the second on the other, each labelled with its conditional probability. Multiplying along a path gives the probability of that path, and the paths that end in the same event add up to its total probability. This is the law of total probability:

Take a screening test for a condition that 1 % of the people tested have: its prevalence. The test is positive for 95 % of the people with the condition, its sensitivity, and negative for 90 % of those without it, its specificity. The chance of a positive result is \(0.01 \times 0.95 + 0.99 \times 0.1 = 0.1085\).

Bayes' rule

The test's sensitivity answers the question "if you have the condition, how likely is a positive result?". A patient with a positive result wants the reverse: "given a positive result, how likely is it that I have the condition?". Bayes' rule turns one into the other:

For the screening test, \(P(\text{condition} \mid \text{positive}) = \tfrac{0.95 \times 0.01}{0.1085} = 0.0876\). Fewer than 9 in 100 of the people who test positive have the condition, although the test finds 95 % of the cases.

Natural frequencies

The result surprises almost everyone, and it is easier to see with counts than with probabilities. Picture 10,000 people taking the test:

Of the 1,085 positives, 95 have the condition: \(\tfrac{95}{1085} = 0.0876\), the same answer. The false positives win because they are a small share (10 %) of a very large group, while the true positives are a large share of a very small one. That is the base-rate effect: the rarer the condition, the less a positive result means.

The answer depends strongly on the prevalence. At 1 % it is 0.0876; if one person in ten had the condition, the same test would give 0.5135.

Fraud alerts

A bank's fraud model raises an alert on 96 % of fraudulent card payments and on 2 % of genuine ones, and 0.5 % of payments are fraudulent. Out of 10,000 payments, 50 are fraudulent and 48 of them raise an alert; of the 9,950 genuine payments, 199 raise one too. So \(P(\text{fraud} \mid \text{alert}) = \tfrac{48}{247} = 0.1943\): four alerts in five are false alarms. The model is still useful, since an alert makes fraud almost 39 times as likely as the base rate of 0.5 %, but an investigator who treats every alert as fraud will be wrong most of the time.

In code