ML Math

One query and ten keys of dimension 512. Without scaling, the dot products are large and the softmax puts 99.6% of the weight on a single key — attention becomes a hard lookup with almost no gradient for the rest. Dividing by √512 brings the largest weight down to 21%. The √d in the formula is there for exactly this reason.

• Attention weights are not explanations; high weight on a token does not mean it caused the output.
• Full attention costs grow with the square of the sequence length.
• Attention has no sense of order by itself; position encodings supply it.

rng = np.random.default_rng(6)
softmax = lambda z: np.exp(z - z.max()) / np.exp(z - z.max()).sum()
d = 512
q, K = rng.normal(size=d), rng.normal(size=(10, d))   # one query, ten keys
raw = softmax(K @ q).max()                             # unscaled scores grow with sqrt(d)
scaled = softmax(K @ q / np.sqrt(d)).max()

Assumes import numpy as np. Each snippet carries its own example numbers; the comments say which are made up or simulated.