Analytics✓ Mathematical
◆ The PatternThe first hour with any dataset — a repeatable profiling workflow
EDA is the disciplined version of "poking at the data": shape and types first, then missingness, then each variable's distribution, then relationships between variables. The goal is not pretty plots — it is a list of surprises and decisions: columns to fix, outliers to investigate, transformations to apply, hypotheses worth testing.
shape → types → missing → distributions → relationships
The EDA loop — always in this order. Each step decides what the next one means.
// Interactive — skew the data and inject outliers, watch mean vs median diverge
Skew0.40
Outliers %0%
Mean—
Median—
| Step | pandas | You're looking for |
|---|---|---|
| Shape & types | df.shape, df.info() | Row count, wrong dtypes, IDs read as numbers |
| Missingness | df.isna().mean() | Columns to drop, impute, or flag — see missing data |
| Numeric profile | df.describe() | Impossible values, skew (mean ≠ median), scale |
| Categorical profile | value_counts() | Cardinality, typos ("NY" vs "N.Y."), rare levels |
| Distributions | df.hist(), boxplot | Shape, outliers — see distribution shape |
| Relationships | df.corr(), scatter_matrix | Redundancy, leakage candidates, target signal |
# Python — the 10-line EDA opener import pandas as pd df = pd.read_csv('data.csv') print(df.shape) print(df.info()) # dtypes + non-null counts print(df.describe(include='all').T) # numeric + categorical profile print(df.isna().mean().sort_values(ascending=False).head(10)) for col in df.select_dtypes('object'): print(col, df[col].nunique(), df[col].value_counts().head(3).to_dict()) df.hist(bins=40, figsize=(12, 8)) # every numeric at once
Mean vs median is a free skew detector: when they disagree badly (as in the visual above), the distribution is skewed or contaminated by outliers — report medians, and consider a log transform before modeling.
EDA before splitting? Careful. Profile structure on everything, but tune decisions (imputation values, outlier caps, transformations) on the training split only — otherwise test-set information leaks into your pipeline.
Try it on real data
Kaggle: Titanic (classic messy mix of numeric, categorical, and missing)
UCI: Adult Income (48K rows, categorical-heavy, "?" as missing marker)
Adult hides its missing values as the string "?" — a perfect first test of whether your EDA actually catches what df.isna() misses.
When to use this
✓ Use when: Every new dataset, every refreshed pipeline, before every model. Ten minutes of EDA routinely saves days of debugging silent data problems.
✗ Skip when: Never entirely — but for a stable, monitored pipeline the ongoing version of EDA is drift detection, not manual re-profiling.
Quick start — copy to notebook
pip install pandas seaborn matplotlib
# ────────────────────────────────────────
import seaborn as sns
import matplotlib.pyplot as plt
df = sns.load_dataset('titanic')
print(df.shape)
print(df.describe(include='all').T)
print(df.isna().mean().sort_values(ascending=False).head())
df.hist(bins=40, figsize=(12, 6))
sns.heatmap(df.isna(), cbar=False) # missingness at a glance
plt.show()