Probability✓ Mathematical
◆ The PatternMaximum Likelihood Estimation — why MSE and Cross-Entropy exist
MLE asks: what parameters make the observed data most probable? Most ML training objectives are secretly MLE under a particular assumed distribution.
θ* = argmax_θ Σ log p(xᵢ | θ)
Maximise log-likelihood (summing logs avoids numerical underflow with tiny probabilities)
Gaussian: p(x|μ,σ) = (1/√2πσ²) · exp(−(x−μ)²/2σ²)
MLE on Gaussian noise assumption → MSE loss. MLE on Bernoulli → Cross-Entropy loss.
// Gaussian distribution — adjust mean and variance
Mean μ0.0
Std σ1.0
Key insight: Training with MSE loss = assuming your errors are Gaussian distributed. Training with Cross-Entropy = assuming Bernoulli/Categorical outputs. The loss function encodes your distributional assumption.
Pattern bridge: Maximum likelihood estimation is the mathematical foundation of fitting a normal distribution to data. The same principle drives volatility estimation in markets — finding the parameters that best explain observed returns.