Lesson 08 RAG Basics
Theme:

Homework

Three tasks extending the RAG pipeline to your own data and exploring the impact of chunking and metadata strategies.

Complete all three lab exercises before starting the homework. These tasks build on the RAG pipeline you implemented in the lab.

📥 Reference Solution

⬇ lab_answer.pdf

Task 1 — RAG Over Your Own Documents

Objective

Apply the RAG pipeline to a document collection you care about — personal notes, work documentation, a textbook chapter, or any text corpus.

Instructions

  1. Collect at least 10 documents (or chunk a larger document into 10+ pieces)
  2. Index them using the pipeline from the lab
  3. Write 10 questions that can be answered from your documents
  4. Run each question through your rag_answer() function
  5. Rate each answer on a 1–5 scale:
    • 5 — Perfect, fully grounded in context
    • 4 — Good, minor issues
    • 3 — Acceptable but missing key details
    • 2 — Poor, partially incorrect or hallucinated
    • 1 — Wrong or completely ungrounded
  6. Calculate your average score and document patterns in failures

Deliverable

A notebook with your 10 QA pairs, ratings, average score, and a paragraph analysing retrieval quality.

Task 2 — Chunk Size Experiment

Objective

Empirically measure how chunk size affects retrieval quality in your RAG pipeline.

Instructions

  1. Take a longer document (at least 2000 words) and chunk it three ways:
    • 128 tokens per chunk (with 20-token overlap)
    • 256 tokens per chunk (with 40-token overlap)
    • 512 tokens per chunk (with 80-token overlap)
  2. Create a separate ChromaDB collection for each chunk size
  3. Write 5 test questions and run them against all three collections
  4. For each question, compare the retrieved chunks across the three sizes
  5. Document: which chunk size produced the most relevant retrievals? Were there trade-offs?

Deliverable

A notebook with the three collections, 5 test queries, retrieved results side by side, and a written comparison.

Task 3 — Date Metadata Filtering

Objective

Add temporal metadata to your documents and implement date-range filtering in queries.

Instructions

  1. Add a date metadata field to each document (as an ISO string or Unix timestamp)
  2. Implement a query function that accepts date_from and date_to parameters
  3. Use ChromaDB's where with $gte and $lte operators for date filtering
  4. Test with queries like "What happened last month?" filtered to the appropriate date range
  5. Compare results with and without date filtering

Deliverable

A notebook demonstrating date-filtered RAG queries with comparison to unfiltered results.

Tip: ChromaDB metadata values must be strings, integers, floats, or booleans. For dates, use integer timestamps (e.g., int(datetime.timestamp())) for easy comparison with $gte/$lte.

Submission Checklist