Lesson 09 — RAG Advanced

Lesson 09 · Slide 1

Advanced RAG

Naive RAG pipelines hit a ceiling quickly. This lesson covers techniques to push retrieval quality significantly higher: hybrid search, reranking, query expansion, and evaluation.

Learning Objectives

Identify limitations of naive RAG and when they matter
Implement hybrid search combining BM25 and dense retrieval
Apply cross-encoder reranking to improve precision
Use query expansion and HyDE to improve recall
Evaluate RAG pipelines with RAGAS-style metrics

Lesson 09 · Slide 2

Naive RAG Limitations

The basic embed-and-retrieve pipeline from Lesson 08 has predictable failure modes:

Low recall: Semantic search misses documents that use different vocabulary for the same concept (lexical gap)
Low precision: Top-k results often include marginally relevant documents that dilute the context
Query-document mismatch: Questions and document passages live in different embedding spaces — a question about "fixing bugs" may not match a passage about "debugging techniques"
Single retrieval strategy: No single similarity metric works well for all query types

The 80/20 rule of RAG: Getting a basic pipeline working takes 20% of the effort. Getting it to work well on edge cases takes the remaining 80%.

Lesson 09 · Slide 3

Hybrid Search — BM25 + Dense

Combine the strengths of two retrieval methods: BM25 (keyword/lexical) and dense retrieval (semantic embeddings).

BM25 (Lexical)

Matches exact keywords and phrases
Great for proper nouns, acronyms, rare terms
Fast, no GPU needed
Fails on synonyms and paraphrases

Dense (Semantic)

Understands meaning, not just words
Handles synonyms and paraphrases well
Requires embedding model
Fails on exact keyword matches sometimes

Reciprocal Rank Fusion (RRF)

Merge the two ranked lists using RRF: score(d) = sum(1 / (k + rank_i(d))) where k=60 is standard. Documents that rank high in both lists get the highest fused score.

Lesson 09 · Slide 4

Reranking

After initial retrieval (which must be fast), use a more powerful model to reorder the top candidates for precision.

Bi-Encoder (Retrieval)

Encodes query and document independently
Fast: pre-compute all document embeddings
Less accurate: no cross-attention between query and doc

Cross-Encoder (Reranking)

Encodes query and document together
Slow: must run for each (query, doc) pair
More accurate: full attention between query and doc tokens

from sentence_transformers import CrossEncoder
reranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2")
pairs = [(query, doc) for doc in candidate_docs]
scores = reranker.predict(pairs)
# Sort by score descending to get the best matches first

Lesson 09 · Slide 5

Query Expansion & HyDE

Improve recall by reformulating the query before retrieval.

Query Expansion

Use an LLM to generate 2–3 alternative phrasings of the original query. Retrieve with all phrasings and merge results.

# Ask Claude to rephrase the query
expansions = client.messages.create(
    model="claude-haiku-4-5", max_tokens=200,
    messages=[{"role": "user", "content":
        f"Generate 3 alternative phrasings of this search query. "
        f"Return only the 3 queries, one per line.\n\nQuery: {query}"}]
).content[0].text.strip().split("\n")

HyDE (Hypothetical Document Embeddings)

Instead of embedding the question, ask the LLM to generate a hypothetical answer, then embed that. The hypothesis is closer in embedding space to the actual documents than the question is.

Trade-off: Both techniques add an LLM call before retrieval, increasing latency and cost. Use them when recall is more important than speed.

Lesson 09 · Slide 6

Evaluating RAG — RAGAS Metrics

How do you know if your RAG pipeline is actually good? RAGAS (Retrieval Augmented Generation Assessment) defines four key metrics:

Metric	Measures	How
Faithfulness	Is the answer grounded in the retrieved context?	LLM checks each claim against context
Answer Relevancy	Does the answer address the question?	Generate questions from the answer; compare to original
Context Precision	Are the retrieved docs actually relevant?	LLM rates each retrieved chunk
Context Recall	Did we retrieve all the needed information?	Compare retrieved context to ground truth answer

LLM-as-judge: All RAGAS metrics use an LLM to evaluate. This is cost-effective and correlates well with human judgement for most use cases.

Lesson 09 · Slide 7

Demo

Live demonstration: upgrade the Lesson 08 pipeline with hybrid search and reranking.

Demo Steps

Load the same 10 Python docs from Lesson 08
Add BM25 index alongside ChromaDB
Show a query where BM25 finds a result that dense search misses
Implement reciprocal rank fusion
Add cross-encoder reranking on the fused results
Compare answer quality: naive vs hybrid vs hybrid+reranked

Notes: Use a query with a specific term like "lru_cache" that BM25 matches exactly but dense search may rank lower. Show the reranker promoting the most relevant result to position 1.

Lesson 09 · Slide 8

Lab & Homework

Lab Exercises

Hybrid Search — add BM25 with rank_bm25 + reciprocal rank fusion
Reranking — add cross-encoder reranking
Evaluation — LLM-as-judge faithfulness scoring on 5 questions

Homework Tasks

Query expansion: 3 alternative phrasings per question
HyDE: hypothetical document embedding
Build a RAGAS-style eval dataset with 10 QA pairs