Planning✓ Mathematical
◆ The PatternHow much data do you need? Sample size planning for experiments that can actually detect effects
Statistical power is the probability of detecting a real effect if one exists. Convention: aim for 80% power. Power analysis links four quantities — sample size, effect size, significance level, and power — so you can solve for any one given the other three.
Power = 1 − β = P(reject H0 | H1 true)
Power = 0.80 means 80% chance of detecting a real effect. β = Type II error rate.
// Interactive — sample size vs power curve
Effect size d0.50
Alpha0.05
N needed—
# Python — power analysis from statsmodels.stats.power import TTestIndPower analysis = TTestIndPower() # How many samples for d=0.5, power=0.8? n = analysis.solve_power( effect_size=0.5, power=0.8, alpha=0.05 ) print(f"Need {n:.0f} per group") # Power curve import matplotlib.pyplot as plt ns = range(10, 200) powers = [analysis.power(effect_size=0.5, nobs1=n, alpha=0.05) for n in ns] plt.plot(ns, powers); plt.axhline(0.8, ls='--'); plt.show()
Before you experiment: Do the power analysis first. If you need 500 samples per group and can only get 50, the experiment is doomed before it starts.