21 — Core Math

Evaluation Metrics#

EvaluationMathematical
◆ The PatternHow to actually measure if your model is good

Accuracy alone is misleading for imbalanced classes. Precision, Recall, F1 and ROC-AUC give a complete picture.

Precision = TP/(TP+FP)    Recall = TP/(TP+FN)
Precision = of predicted positives, how many real?  |  Recall = of real positives, how many found?
F1 = 2·(Precision·Recall)/(Precision+Recall)
Harmonic mean — punishes models that sacrifice one for the other
// Interactive confusion matrix
True Positives80
False Positives10
False Negatives20
Pattern bridge: Precision and recall trade off like Type I and Type II errors in hypothesis testing. In markets, a MACD signal has the same tradeoff: too sensitive (false positives) vs. too slow (missed moves).
Worked example

1,000 cases: 40 true positives, 10 false positives, 20 missed and 930 true negatives. Accuracy is 97%, mostly from the easy negatives. Precision is 0.80 (four in five alarms are real), recall 0.67 (a third of the positives are missed), and F1 0.73. The metric should match the cost of the mistakes.

Where it misleads
  • F1 weights precision and recall equally; if a miss costs ten times a false alarm, use a weighted measure or the costs directly.
  • Precision depends on how common positives are; the same model has different precision in a different population.
  • A single threshold hides the trade-off; look at the precision–recall curve.
In code
tp, fp, fn, tn = 40, 10, 20, 930
precision = tp / (tp + fp)
recall = tp / (tp + fn)
f1 = 2 * precision * recall / (precision + recall)
accuracy = (tp + tn) / (tp + fp + fn + tn)
Python, in your browser — edit the code and run it again

Assumes import numpy as np. Each snippet carries its own example numbers; the comments say which are made up or simulated.

Sources
  • D. M. W. Powers, “Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness and Correlation”, Journal of Machine Learning Technologies 2(1), 2011
  • T. Saito & M. Rehmsmeier, “The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets”, PLoS ONE 10(3), 2015

Last reviewed October 2026 · How this site checks its content · Report a mistake

Linked from 3
← Previous
Cross-Validation
Open in the full reader, with the topic sidebar