23 — Governance & Trust

Fairness Audits#

Ethics✓ Mathematical
◆ The PatternTesting for bias across protected groups

Fairness audits measure whether your model treats different groups equitably. Demographic parity checks if positive rates are equal across groups. Equalized odds checks if error rates are equal. No single metric captures all fairness — you must choose which definition matches your context.

Disparate Impact = P(ŷ=1|G=a) / P(ŷ=1|G=b)
A ratio below 0.8 (the "four-fifths rule") suggests adverse impact against group b.
// Interactive — fairness metrics across groups
# Python — Fairlearn fairness assessment
from fairlearn.metrics import MetricFrame
from sklearn.metrics import accuracy_score

mf = MetricFrame(
    metrics=accuracy_score,
    y_true=y_test,
    y_pred=y_pred,
    sensitive_features=demographics
)
print(mf.by_group)
print("Ratio:", mf.ratio())
Pattern bridge: Fairness audits apply confidence intervals per subgroup — the same statistical rigour, applied to equity. In markets, behavioral bias recognition teaches the same lesson: your defaults aren't neutral.
← Previous
Lineage Tracking
Open in the full reader, with the topic sidebar →