02 — Evaluate Your Model

ROC & AUC Curves#

Classification✓ Mathematical
◆ The PatternVisualising model performance across all thresholds

The ROC curve plots True Positive Rate vs False Positive Rate at every threshold. The AUC (area under curve) collapses this into a single number: 1.0 = perfect, 0.5 = random. It's threshold-independent, making it ideal for comparing models.

TPR = TP / (TP + FN)    FPR = FP / (FP + TN)
TPR = sensitivity/recall  |  FPR = 1 − specificity  |  plotted as (FPR, TPR)
AUC = ∫ TPR(FPR) dFPR ∈ [0, 1]
AUC = probability that the model ranks a random positive higher than a random negative
// Interactive ROC — adjust model separability
Separability0.65
AUC
# Python — ROC & AUC
from sklearn.metrics import roc_curve, roc_auc_score
import matplotlib.pyplot as plt

fpr, tpr, thresholds = roc_curve(y_true, y_prob)
auc = roc_auc_score(y_true, y_prob)

plt.plot(fpr, tpr, label=f'AUC = {auc:.3f}')
plt.plot([0,1],[0,1], '--', color='gray')
plt.xlabel('FPR'); plt.ylabel('TPR')
plt.legend(); plt.show()
Multi-class: Use roc_auc_score(y, y_prob, multi_class='ovr') with one-vs-rest for multi-class problems.
Pattern bridge: AUC measures discrimination — can the model separate classes? In RSI, you're doing the same thing: separating overbought from oversold regimes across different threshold levels.
← Previous
Confusion Matrix & Classification Metrics
Open in the full reader, with the topic sidebar →