15 — Core Math

Softmax & Probabilities#

Classification✓ Mathematical
◆ The PatternConverting raw scores into a probability distribution

Softmax maps a vector of real-valued logits to probabilities that sum to 1. It amplifies the largest logit, making the winner more decisive.

σ(zᵢ) = e^zᵢ / Σⱼ e^zⱼ
zᵢ = logit for class i  |  numerator = exponentiated score  |  denominator = normalisation
// Adjust logits — watch probabilities redistribute
Temperature scaling: σ(z/T). T<1 → sharper (more confident). T>1 → softer (more uniform). Used in knowledge distillation and language model sampling.
Pattern bridge: Converting logits to probabilities that sum to 1 is a probability distribution in action. Temperature scaling changes the "confidence" — hot = uniform = uncertain market, cold = peaked = consensus.
← Previous
Gradient Clipping
Open in the full reader, with the topic sidebar →