Random variables
PMF, PDF, CDF and percentiles; expected value, variance and standard deviation.
From outcomes to numbers
A random variable gives each outcome of an experiment a number: the face a die shows, the number of units a shop sells in a day, the minutes a call lasts. It is written with a capital letter, \(X\), and a value it takes with a small one, \(x\) or \(k\). Once the outcomes are numbers, we can ask for averages, spreads and percentiles.
A random variable is discrete when its values can be listed one by one: 0, 1, 2 and so on. The list may be finite, like a die's six faces, or go on without end, like the number of customers who walk in during an hour, which has no fixed upper limit: discrete means countable, not finite. A random variable is continuous when it can take any value in an interval, such as a delivery time of 2.37 hours, so its values cannot be listed.
The probability mass function
A discrete random variable is described by its probability mass function (PMF), \(f(k) = P(X = k)\): the probability of each value. The probabilities lie between 0 and 1 and add up to 1. A small shop has recorded how many units of a popular gadget it sells in a day:
| Units sold, k | 0 | 1 | 2 | 3 | 4 | 5 |
|---|---|---|---|---|---|---|
| P(X = k) | 0.05 | 0.15 | 0.3 | 0.25 | 0.15 | 0.1 |
P(X ≥ 3) = 0.5000
The cumulative distribution function
The cumulative distribution function (CDF), \(F(x) = P(X \le x)\), adds up the probabilities from the left. For the shop, \(F(2) = 0.05 + 0.15 + 0.3 = 0.5\): on half of the days it sells 2 units or fewer. The CDF answers every "at most" question directly, and the others through it:
The last two hold for counts, where a single value carries probability. For a continuous variable, \(P(X < x) = P(X \le x) = F(x)\).
Percentiles: the CDF turned around
The CDF turns a value into a probability. The percent point function (PPF), also called the quantile function, turns a probability back into a value: the \(q\)-quantile is the smallest value whose cumulative probability reaches \(q\). The shop's 90th percentile is 4 units, because \(F(3) = 0.75\) falls short of 0.9 and \(F(4) = 0.9\) reaches it.
Which function answers which question
Each of a retailer's questions calls for one function. With \(X\) the number of units sold in a day:
| Question | Function | Answer |
|---|---|---|
| How likely is a day with exactly 3 sales? | PMF: P(X = 3) | 0.25 |
| How likely is a day with no more than 2 sales? | CDF: F(2) | 0.5 |
| How likely is a day with more than 3 sales? | 1 − CDF: 1 − F(3) | 0.25 |
| How many units cover the demand on 90 % of days? | PPF: F⁻¹(0.9) | 4 |
| What it computes | Python (scipy.stats) | Excel |
|---|---|---|
| The distribution: values in A2:A7, probabilities in B2:B7 | sales = rv_discrete(values=([0, 1, 2, 3, 4, 5], [0.05, 0.15, 0.3, 0.25, 0.15, 0.1])) |
=SUM(B2:B7) |
| P(X = 3) | sales.pmf(3) |
=SUMIFS(B2:B7, A2:A7, 3) |
| P(X ≤ 2) | sales.cdf(2) |
=SUMIFS(B2:B7, A2:A7, "<=2") |
| P(X > 3) | sales.sf(3) |
=SUMIFS(B2:B7, A2:A7, ">3") |
| The 90th percentile, with the running totals of B in C2:C7 | sales.ppf(0.9) |
=INDEX(A2:A7, MATCH(TRUE, C2:C7>=0.9, 0)) |
| The mean | sales.mean() |
=SUMPRODUCT(A2:A7, B2:B7) |
| The standard deviation | sales.std() |
=SQRT(SUMPRODUCT((A2:A7-2.6)^2, B2:B7)) |
rv_discrete comes from scipy.stats, like the named distributions; it takes the values and their probabilities.
Expected value
The expected value or mean, \(E[X]\) or \(\mu\), is the average of the values weighted by their probabilities. It is the long-run average over many repetitions, and not necessarily a value \(X\) can take.
A die's expected value is \(\tfrac{1 + 2 + \dots + 6}{6} = 3.5\), a face no die shows. The shop's is \(0 \times 0.05 + 1 \times 0.15 + 2 \times 0.3 + 3 \times 0.25 + 4 \times 0.15 + 5 \times 0.1 = 2.6\) units a day.
Variance and standard deviation
The variance measures the spread around the mean: the expected squared distance from it. Its square root, the standard deviation \(\sigma\), is in the units of \(X\).
For the shop, \(E[X^2] = 8.5\), so \(\operatorname{Var}(X) = 8.5 - 2.6^2 = 1.74\) and \(\sigma = 1.3191\) units. For a die, \(\operatorname{Var}(X) = 2.9167\) and \(\sigma = 1.7078\).
Linearity of expectation
Scaling and shifting a random variable scales and shifts its mean. The spread only scales, because a shift moves every value by the same amount:
If each gadget earns the shop a margin of 20 and the display costs 15 a day, the daily profit is \(20X - 15\), with mean \(20 \times 2.6 - 15 = 37\) and standard deviation \(20 \times 1.3191 = 26.3818\). Means also add: the mean of a sum is the sum of the means, whether or not the variables are independent.
Densities for continuous variables
A continuous variable has no probability at any single value, so it is described by a probability density function (PDF), \(f(x)\), whose area gives the probability: \(P(a \le X \le b) = \int_a^b f(x)\,dx\). A density is probability per unit of \(x\), not a probability, so it can be greater than 1 (chapter 4 shows one). A chart of a density is labelled "density", never "probability".
Quiz
Three questions on the chapter. Your best score is kept in this browser.