Data AnalysisStatistical
1 min read
◆ The PatternSkewness, kurtosis, QQ plots — is your data normal, and does it matter?
Before modelling, know your data's shape. Skewness measures asymmetry (are the tails lopsided?), kurtosis measures tail heaviness (how many extreme values?), and QQ plots show deviations from normality at a glance.
Skew = E[(X−μ)³] / σ³
Skew = 0 is symmetric | >0 right tail | <0 left tail
Kurt = E[(X−μ)⁴] / σ⁴ − 3
Excess kurtosis = 0 is normal | >0 heavy tails | <0 light tails
// Interactive — adjust skew and kurtosis
Skewness0.0
Tail weight3.0
When it matters: Linear regression’s tests and intervals assume roughly normal residuals (the fit itself does not). Many tests assume normality. Log-transform right-skewed data. Market returns have heavy tails (excess kurtosis) — never assume normal.
Pattern bridge: Fat tails are why risk models that assume a bell curve underestimate the bad days — see tail risk and expected shortfall, which looks at how bad the worst cases are rather than just where they start.
Worked example
10,000 simulated incomes from a lognormal distribution: skewness 3.57, a mean of 49,351 and a median of 36,160. The “average” earner earns less than the average. On a log scale the skewness is 0.0: the data are symmetric in ratios, which is often the right scale to model them on.
Where it misleads
- Skewness and kurtosis estimates are dominated by a few extreme points and are unstable in small samples.
- Normality tests reject trivial departures in large samples and miss big ones in small samples; look at a QQ plot instead.
- Transforming the target changes what a model predicts (a median rather than a mean on the original scale).
In code
from scipy.stats import skew rng = np.random.default_rng(8) income = rng.lognormal(mean=10.5, sigma=0.8, size=10_000) # right-skewed, like incomes stats_raw = (round(skew(income), 2), round(income.mean()), round(np.median(income))) skew_log = round(skew(np.log(income)), 2)
Assumes import numpy as np and import pandas as pd (and SciPy where imported). Each snippet simulates or makes up its own data, as the comments say.
The same idea with a library
# Python — distribution diagnostics from scipy.stats import skew, kurtosis, probplot import matplotlib.pyplot as plt print(f"Skew: {skew(data):.3f}") print(f"Kurt: {kurtosis(data):.3f}") # QQ plot — points on line = normal fig, ax = plt.subplots() probplot(data, plot=ax) plt.show()
Sources
- D. N. Joanes & C. A. Gill, “Comparing Measures of Sample Skewness and Kurtosis”, Journal of the Royal Statistical Society D 47(1), 1998
- G. E. P. Box & D. R. Cox, “An Analysis of Transformations”, Journal of the Royal Statistical Society B 26(2), 1964
Last reviewed October 2026 · How this site checks its content · Report a mistake
Linked from 19
- Pattern EssaysThe Bell in EverythingWherever many small, independent forces combine, the same shape appears
- Pattern EssaysThe Feedback LoopWhen a system’s output feeds back into its input, small nudges can cascade into enormous change
- Pattern EssaysThe FractalZoom in and the pattern repeats, the same shape living inside itself
- Chart PatternsDojiOpen ≈ Close — the market is undecided
- Chart PatternsEvening StarLong green → small star → long red — dusk after daylight
- Chart PatternsHammerLong lower shadow, small body at top — buyers reclaimed
- Chart PatternsHead & ShouldersThree peaks — the classic top reversal
- Chart PatternsInverse Head & ShouldersThe mirror image — three troughs signaling a bottom
- Chart PatternsRectangleHorizontal support and resistance — a consolidation box
- Market PsychologyFraming EffectDecisions change based on how information is presented.
- Market PsychologyLoss AversionLosses hurt ~2× more than equivalent gains feel good (Kahneman & Tversky, Prospect Theory).
- Risk & PortfolioExpected Shortfall (CVaR)Average loss in the worst α % of scenarios
- Risk & PortfolioKelly CriterionOptimal bet size for maximum geometric growth
- Risk & PortfolioTail RiskExtreme events beyond normal-distribution assumptions
- ML MathMLE & GaussianMaximum Likelihood Estimation — why MSE and Cross-Entropy exist
- MLOps & Production MLLatency & Throughputp50, p99, batching, and the tradeoffs that define your SLA
- The ToolkitCentral Limit Theorem & SamplingWhy averages become normal — the theorem underneath every CI and t-test
- The ToolkitExploratory Data Analysis (EDA)The first hour with any dataset — a repeatable profiling workflow
- Timeseries EngineeringStationarityThe assumption that makes forecasting possible