29 — Statistical Foundations

Central Limit Theorem & Sampling#

StatisticsMathematical 1 min read
◆ The PatternWhy averages become normal — the theorem underneath every CI and t-test

Take any population — skewed, lumpy, weird. Draw samples of size n and compute each sample's mean. The Central Limit Theorem says the distribution of those means approaches a normal distribution as n grows, regardless of the population's shape. That single fact is why confidence intervals, t-tests, and A/B testing work at all.

x̄  →  N(μ, σ²/n)
Sampling distribution of the mean: centred on the true mean μ, with variance shrinking as 1/n.
SE = σ / √n
Standard error = the standard deviation of the sample mean. Quadruple the sample to halve the error.
// Interactive — skew the population, grow n, watch the sample means turn normal
Population skew0.70
Sample size n5
SE ∝ 1/√n—
ConceptWhat it saysPractical use
Law of Large Numbersx̄ → μ as n growsMore data → estimates converge
Central Limit Theoremx̄ becomes normally distributedJustifies CIs, t-tests, z-tests
Standard errorSpread of x̄ is σ/√nSizing experiments, error bars
Rule of thumb: n ≥ 30 usually suffices for the normal approximation — but the more skewed the population, the more n you need. Very heavy-tailed data (rare in nature, common in finance) converges slowly or, with infinite variance, not at all.
Pattern bridge: The CLT also assumes independent draws. Market returns are autocorrelated and fat-tailed — see distribution shape and Monte Carlo simulation for why financial error bars deserve extra suspicion.
When to use this
✓ Use when: Reasoning about any average, proportion, or error bar — experiment metrics, survey estimates, batch statistics in monitoring.
✗ Skip when: Data is strongly dependent (time series), heavy-tailed, or you care about extremes rather than means — the CLT says nothing about the tails of the original distribution.
Quick start — copy to notebook
pip install numpy matplotlib
# ────────────────────────────────────────
import numpy as np
import matplotlib.pyplot as plt

population = np.random.exponential(1.0, 1_000_000)

fig, axes = plt.subplots(1, 3, figsize=(12, 3), sharex=True)
for ax, n in zip(axes, [1, 5, 30]):
    idx = np.random.randint(0, len(population), (10_000, n))
    ax.hist(population[idx].mean(axis=1), bins=60)
    ax.set_title(f"sample means, n = {n}")
plt.tight_layout(); plt.show()
Worked example

Means of exponential samples: the skewness is 1.99 for single values, 0.87 for means of 5 and 0.35 for means of 30 — falling as 2/√n, the CLT at work. Cauchy data never get there: the spread (interquartile range) of a single draw is 2.10, and of the mean of 1,000 draws still 2.06. Without a finite variance, averaging does not help at all.

Where it misleads
  • “n = 30 is enough” is a rule of thumb that fails for strongly skewed or heavy-tailed data.
  • The CLT is about the mean; medians, maxima and ratios converge differently, or not at all.
  • Dependent observations converge much more slowly than independent ones.
In code
from scipy.stats import skew
rng = np.random.default_rng(20)
skew_of_mean = {n: round(skew(rng.exponential(1, (20_000, n)).mean(axis=1)), 2) for n in (1, 5, 30)}
cauchy = rng.standard_cauchy((2_000, 1_000))
spread = (round(np.subtract(*np.percentile(cauchy[:, 0], [75, 25])), 2),          # one draw
          round(np.subtract(*np.percentile(cauchy.mean(axis=1), [75, 25])), 2))   # mean of 1,000
Python, in your browser — edit the code and run it again

Assumes import numpy as np and import pandas as pd (and SciPy where imported). Each snippet simulates or makes up its own data, as the comments say.

The same idea with a library
# Python — watch the CLT happen
import numpy as np
import matplotlib.pyplot as plt

# A very non-normal population (exponential)
population = np.random.exponential(scale=1.0, size=1_000_000)

for n in [1, 5, 30]:
    means = [np.mean(np.random.choice(population, n))
             for _ in range(10_000)]
    plt.hist(means, bins=60, alpha=0.5, label=f"n={n}")

plt.legend(); plt.title("Sampling distribution of the mean")
plt.show()

# Standard error shrinks as 1/sqrt(n)
print(population.std() / np.sqrt(30))
Sources
  • An Introduction to Probability Theory and Its Applications, Vol. 2 (2nd ed.), W. Feller, Wiley, 1971
  • Introduction to Robust Estimation and Hypothesis Testing (4th ed.), R. R. Wilcox, Academic Press, 2017

Last reviewed October 2026 · How this site checks its content · Report a mistake

Linked from 7
← Previous
Choosing the Right Statistical Test
Open in the full reader, with the topic sidebar