02 — Foundations

Linear Regression#

Supervised✓ Mathematical
◆ The PatternFitting a line through data — the simplest predictive model

Linear regression finds the best-fit line through data. Given input x, we predict ŷ by learning weight w (slope) and bias b (intercept) that minimise prediction error.

ŷ = w · x + b
ŷ = prediction  |  w = weight  |  x = input  |  b = bias
MSE = (1/n) · Σ (yᵢ − ŷᵢ)²
MSE = Mean Squared Error loss  |  minimised by gradient descent
// Interactive — drag sliders to fit the line
Weight w0.50
Bias b0.00
MSE
OLS Solution: For simple linear regression, the optimal weights have a closed-form: w = Σ(xᵢ−x̄)(yᵢ−ȳ) / Σ(xᵢ−x̄)². Gradient descent finds the same answer iteratively.
Pattern bridge: Fitting a line is the same act everywhere — linear regression in statistics, simple moving averages in markets.
← Previous
Vectors & Matrices
Open in the full reader, with the topic sidebar →