Optimization✓ Mathematical
◆ The PatternMeasuring how wrong the model is — the objective being minimised
The loss function defines what the model is optimising. Different tasks need different loss functions. Getting this wrong is one of the most common ML mistakes.
MSE = (1/n)·Σ(y−ŷ)²
Regression — squared penalty, sensitive to outliers
MAE = (1/n)·Σ|y−ŷ|
Robust regression — linear penalty, outlier-resistant
CE = −Σ yᵢ·log(ŷᵢ)
Classification — penalises confident wrong predictions exponentially
Huber = { ½(y−ŷ)² if |y−ŷ|≤δ, δ|y−ŷ|−½δ² otherwise }
Huber — smooth MSE near zero, MAE for large errors. Best of both.
// MSE vs MAE vs Huber — drag to see how penalty scales with error
Error magnitude1.00
MSE1.000
MAE1.000
Pattern bridge: Loss functions measure distance from truth — like variance measures distance from the mean.