28 — Statistical Foundations

Choosing the Right Statistical Test#

Statistics✓ Mathematical
◆ The Patternt-test, chi-square, ANOVA and their nonparametric cousins — a decision map

Most real questions reduce to three: how many groups am I comparing, are the measurements paired, and can I assume normality? Answer those and the test picks itself. Parametric tests (t-test, ANOVA) assume roughly normal data and are more powerful; nonparametric tests (Mann-Whitney, Wilcoxon, Kruskal-Wallis) rank the data instead and work almost anywhere.

t = (x̄₁ − x̄₂) / √(s₁²/n₁ + s₂²/n₂)
Two-sample t (Welch) = difference in means scaled by its standard error. Default choice for two independent groups.
χ² = Σ (observed − expected)² / expected
Chi-square = for counts and categories — is the observed table compatible with independence?
// Interactive decision map — set your situation, the recommended test lights up
Groups2
PairedNo
Normal-ishYes
Use
QuestionParametricNonparametric
2 independent groups differ?Welch t-testMann-Whitney U
Before vs after (paired)?Paired t-testWilcoxon signed-rank
3+ groups differ?One-way ANOVAKruskal-Wallis
Two categorical variables related?Chi-square test of independence
Two numeric variables related?Pearson rSpearman ρ
Sample from this distribution?Kolmogorov-Smirnov / Shapiro-Wilk
# Python — the whole decision map in scipy
from scipy import stats

# 2 independent groups
stats.ttest_ind(a, b, equal_var=False)   # Welch t-test
stats.mannwhitneyu(a, b)                    # nonparametric

# Paired (same subjects, before/after)
stats.ttest_rel(before, after)
stats.wilcoxon(before, after)               # nonparametric

# 3+ groups
stats.f_oneway(g1, g2, g3)                  # ANOVA
stats.kruskal(g1, g2, g3)                   # nonparametric

# Categorical vs categorical
chi2, p, dof, exp = stats.chi2_contingency(contingency_table)

# Check normality first (n < ~50)
stat, p = stats.shapiro(a)
Normality worries less than you think: with n > ~30 per group, the Central Limit Theorem makes the t-test robust to non-normal data. Reach for nonparametric tests when samples are small, heavily skewed, or ordinal.
ANOVA says "some group differs" — not which. Follow a significant ANOVA with pairwise post-hoc tests (Tukey HSD) rather than running raw t-tests between every pair, which inflates the false-positive rate.
When to use this
Use when: You have a concrete comparison question and need the defensible test — experiments, feature launches, model comparisons, survey analysis.
Skip when: The dataset is the full population, or you need effect estimates rather than yes/no answers — estimate with bootstrap or regression instead.
Try it on real data
Palmer Penguins (3 species — made for ANOVA and chi-square) UCI: Wine Quality (rating groups, skewed features)
Penguins ships with seaborn (sns.load_dataset('penguins')) — three species, numeric and categorical columns, a few NaNs. Every test in the table above has a natural question here.
Quick start — copy to notebook
pip install scipy pandas seaborn
# ────────────────────────────────────────
import pandas as pd
import seaborn as sns
from scipy import stats

df = sns.load_dataset('penguins').dropna()
groups = [g for _, g in df.groupby('species')['body_mass_g']]

print(stats.f_oneway(*groups))     # 3 groups, parametric
print(stats.kruskal(*groups))      # 3 groups, rank-based

chi2, p, dof, _ = stats.chi2_contingency(
    pd.crosstab(df['species'], df['island']))
print(f"chi2 = {chi2:.1f}, p = {p:.4g}")
← Previous
Hypothesis Testing & p-values
Open in the full reader, with the topic sidebar →