Retrieval-Augmented Generation✓ Mathematical
◆ The PatternGrounding LLM answers in your own documents
RAG solves the hallucination problem by retrieving relevant documents before generation. The pipeline: Chunk documents → Embed chunks → Store in vector DB → At query time: Embed query → Retrieve top-K chunks → Generate answer with chunks as context.
Pipeline: Query → Embed → Search → Rerank → Augment Prompt → Generate
Each stage has design choices: chunk size, overlap, embedding model, retriever, reranker, prompt template.
Chunk size: 256–512 tokens with 10–20% overlap
Too small → lost context. Too large → diluted relevance. Semantic chunking (split at paragraph breaks) works better than fixed-size.
Common pitfalls: (1) Chunking destroys context — tables, lists split mid-content. (2) Embedding model mismatch — query embeddings must match document embeddings. (3) Top-K too small — misses relevant but lower-ranked chunks. (4) No reranking — embedding similarity ≠ answer relevance.
Reranking is the highest-impact improvement for most RAG systems. A cross-encoder reranker on top-20 retrieval results can boost precision@5 by 15–20%.
Interactive — RAG pipeline flow
Python — RAG with LangChain#
from langchain_community.vectorstores import FAISS
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.chains import RetrievalQA
# 1. Chunk documents
splitter = RecursiveCharacterTextSplitter(
chunk_size=500, chunk_overlap=50)
chunks = splitter.split_documents(documents)
# 2. Embed & store
vectorstore = FAISS.from_documents(chunks, OpenAIEmbeddings())
# 3. Retrieve & generate
qa = RetrievalQA.from_chain_type(
llm=ChatOpenAI(model="gpt-4o", temperature=0),
retriever=vectorstore.as_retriever(search_kwargs={"k": 5}),
)
answer = qa.invoke("What is the refund policy?")Pattern bridge: Retrieval-Augmented Generation grounds the model in external knowledge. Cosine similarity retrieves relevant passages. In statistics, Bayesian updating brings prior evidence to new questions.