Vector DB · ANN · HNSW✓ Mathematical
◆ The PatternFinding semantically similar content at scale
Embedding search converts text to vectors and finds nearest neighbors. The embedding model maps text to a dense vector (768–1536 dimensions). Cosine similarity or dot product measures closeness. For millions of vectors, exact search is too slow — we use Approximate Nearest Neighbors (ANN).
cosine_sim(a, b) = (a · b) / (‖a‖ · ‖b‖) ∈ [−1, 1]
1 = identical direction, 0 = orthogonal, −1 = opposite. Most embedding models are L2-normalized.
HNSW: hierarchical navigable small world graph
Multi-layer graph: top layers for coarse search, bottom layers for precise. O(log n) query time, ~95%+ recall.
Vector databases: Pinecone (managed), Qdrant (open-source), pgvector (PostgreSQL), Chroma (lightweight), Weaviate (hybrid search). For smaller datasets (<100K vectors), brute-force exact search in FAISS is fast enough.
Hybrid search (BM25 keyword + semantic embedding) consistently outperforms either alone. Most production systems combine both with reciprocal rank fusion.
Interactive — vector space nearest neighbor search
Python — embedding search with FAISS#
import faiss
import numpy as np
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("BAAI/bge-base-en-v1.5")
# Index documents
docs = ["Machine learning basics", "Neural networks intro", ...]
embeddings = model.encode(docs, normalize_embeddings=True)
dim = embeddings.shape[1]
index = faiss.IndexFlatIP(dim) # inner product (= cosine for normalized)
index.add(embeddings.astype('float32'))
# Search
query = model.encode(["How do neural nets work?"],
normalize_embeddings=True)
scores, indices = index.search(query.astype('float32'), k=5)
for i, (score, idx) in enumerate(zip(scores[0], indices[0])):
print(f"{i+1}. [{score:.3f}] {docs[idx]}")Pattern bridge: Finding nearest neighbors in vector space is cosine similarity at scale. In statistics, k-nearest-neighbors in feature space is the same idea.