22 — Core Math

Cosine Similarity#

DistanceMathematical 1 min read
◆ The PatternMeasuring angular similarity between vectors — the backbone of retrieval

Cosine similarity measures the angle between two vectors, ignoring magnitude. Two vectors pointing the same direction have similarity 1, opposite = −1, perpendicular = 0. It's the standard metric for embeddings and RAG retrieval.

cos(θ) = (a · b) / (||a|| · ||b||)
a · b = dot product  |  ||a|| = L2 norm  |  Range: [−1, 1]
Cosine Distance = 1 − cos(θ)
Converts similarity to distance  |  Range: [0, 2]  |  0 = identical direction
// Two vectors — adjust angle to see similarity change
Angle between vectors30°
Cosine Sim—
In practice: Embeddings are often L2-normalized, making cosine similarity equivalent to a simple dot product. This is why dot-product search (FAISS, HNSW) is so fast.
Pattern bridge: The angle between embeddings measures semantic similarity — the same geometry as Pearson correlation on centered data. Vector search in RAG relies on this. The dot products and norms it is built from are in Vectors & Matrices.
Worked example

(1, 2, 0) and (2, 4, 0) point the same way: cosine 1.0, although they are 2.24 apart in Euclidean distance. (1, 2, 0) and (0, 0, 3) are perpendicular: cosine 0. Normalise vectors to length 1 first and the dot product is the cosine, which is why vector databases store unit vectors.

Where it misleads
  • Cosine ignores length, which sometimes carries meaning (a longer document, a more confident embedding).
  • Similarity scores are only comparable within one embedding model; 0.8 in one model is not 0.8 in another.
  • In high dimensions many unrelated vectors sit at similar cosines, so thresholds need calibrating on real pairs.
In code
a, b = np.array([1.0, 2.0, 0.0]), np.array([2.0, 4.0, 0.0])    # same direction, twice as long
c = np.array([0.0, 0.0, 3.0])
cos = lambda u, v: u @ v / (np.linalg.norm(u) * np.linalg.norm(v))
result = dict(cos_ab=cos(a, b), dist_ab=np.linalg.norm(a - b), cos_ac=cos(a, c))
unit = lambda v: v / np.linalg.norm(v)
dot_of_units = unit(a) @ unit(b)                  # on normalized vectors, dot product = cosine
Python, in your browser — edit the code and run it again

Assumes import numpy as np. Each snippet carries its own example numbers; the comments say which are made up or simulated.

The same idea with a library
# PyTorch cosine similarity
sim = F.cosine_similarity(a, b, dim=-1)          # batch-wise
sim = torch.dot(a, b) / (a.norm() * b.norm())    # single pair

# For nearest-neighbor retrieval:
sims = query @ embeddings.T                       # all similarities at once
top_k = sims.topk(10)                             # top-10 most similar
Sources
  • G. Salton, A. Wong & C. S. Yang, “A Vector Space Model for Automatic Indexing”, Communications of the ACM 18(11), 1975
  • Introduction to Information Retrieval, C. D. Manning, P. Raghavan & H. Schütze, Cambridge University Press, 2008

Last reviewed October 2026 · How this site checks its content · Report a mistake

Check yourself

1. What is the cosine similarity of [1, 2] and [2, 4]?

Show answer

C. The vectors point the same way; cosine similarity ignores length.

2. For vectors normalised to length 1, how do cosine similarity and Euclidean distance compare?

Show answer

C. That is why vector databases can use either once embeddings are normalised.

3. A cosine similarity of 0 means the vectors are…

Show answer

B. Opposite directions give −1. Many embedding spaces rarely produce negative values, so 0 already means “unrelated”.

Linked from 4
← Previous
Eval Metrics
Open in the full reader, with the topic sidebar