RAG Basics

Retrieval-Augmented Generation lets you ground LLM answers in your own data — no fine-tuning required. This lesson covers embeddings, vector stores, and the core RAG pipeline.

Learning Objectives

• Explain why RAG solves knowledge cutoff, private data, and hallucination problems
• Convert text to vectors using embedding models
• Store and query embeddings in ChromaDB
• Build an end-to-end RAG pipeline: chunk → embed → store → retrieve → generate
• Apply effective chunking strategies for retrieval quality

Why RAG?

LLMs have three fundamental limitations that RAG addresses directly:

RAG is the most cost-effective way to give an LLM access to domain-specific knowledge.

Embedding Models

An embedding model converts text into a dense numeric vector that captures semantic meaning. Similar texts produce similar vectors.

• Input: any text string (sentence, paragraph, document chunk)
• Output: a fixed-size float vector (e.g., 384 dimensions for MiniLM)
• Key property: cosine similarity between vectors reflects semantic similarity

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")
embeddings = model.encode(["Python is great", "I love Python programming"])
# embeddings.shape == (2, 384)

Vector Stores — ChromaDB

A vector store indexes embeddings for fast similarity search. ChromaDB is a lightweight, local-first option perfect for prototyping.

import chromadb

chroma_client = chromadb.Client()  # in-memory
collection = chroma_client.create_collection("my_docs")

# Add documents (ChromaDB can auto-embed with a configured model)
collection.add(
    ids=["doc1", "doc2"],
    documents=["Python pathlib guide", "JavaScript async patterns"],
    metadatas=[{"topic": "python"}, {"topic": "javascript"}]
)

# Query
results = collection.query(query_texts=["file handling in Python"], n_results=2)
print(results["documents"])

The RAG Pipeline

The complete pipeline has two phases: indexing (offline) and querying (online).

Indexing Phase

1. Chunk — split documents into retrievable units (256–512 tokens)
2. Embed — convert each chunk to a vector
3. Store — index vectors + metadata in a vector store

Query Phase

1. Embed query — same model as indexing
2. Retrieve — find top-k similar chunks
3. Inject — place retrieved chunks into the prompt as context
4. Generate — Claude answers grounded in the retrieved context

Chunking for Retrieval

How you split documents directly impacts retrieval quality. Chunks that are too large dilute relevance; chunks that are too small lose context.

Overlap Matters

Adding 10–20% overlap between consecutive chunks prevents information from being split across chunk boundaries. A sentence that spans two chunks would otherwise be lost to both.

# Simple chunking with overlap
def chunk_text(text, chunk_size=500, overlap=50):
    words = text.split()
    chunks = []
    for i in range(0, len(words), chunk_size - overlap):
        chunk = " ".join(words[i:i + chunk_size])
        if chunk:
            chunks.append(chunk)
    return chunks

Demo

Live demonstration: build a RAG system over 10 Python documentation files.

Lab & Homework