LangChain & LlamaIndex

LLM orchestration frameworks: when abstraction helps, when it hurts, and how to choose between them and direct API calls.

Learning Objectives

• Understand the purpose of orchestration frameworks and the problems they solve
• Build chains and agents with LangChain using LCEL (LangChain Expression Language)
• Build RAG pipelines with LlamaIndex using document loaders, indexes, and query engines
• Make informed decisions about framework vs. direct API for your projects
• Integrate both frameworks with Claude, OpenAI, and open-source models

Why Orchestration Frameworks?

As LLM applications grow beyond a single API call, you face recurring problems: chaining prompts, managing conversation memory, connecting to external tools, and retrieving context from documents. Orchestration frameworks provide reusable abstractions for these patterns.

LangChain: Core Concepts

LangChain is the most popular LLM orchestration framework. It provides composable building blocks for chaining LLM calls with tools, memory, and retrieval.

Key Components

• Chat Models — Unified interface to Claude, OpenAI, Ollama, and others via ChatAnthropic, ChatOpenAI, etc.
• Prompt Templates — Parameterised prompt construction with variable substitution and message formatting
• Chains (LCEL) — The pipe operator (|) composes components: prompt | model | parser
• Memory — ConversationBufferMemory, ConversationSummaryMemory, window-based memory for multi-turn conversations
• Tools & Agents — Let the LLM decide which tools to call, with built-in and custom tool definitions
• Output Parsers — StrOutputParser, JsonOutputParser, Pydantic-based structured output

from langchain_anthropic import ChatAnthropic
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant that explains {topic} simply."),
    ("human", "{question}")
])
model = ChatAnthropic(model="claude-sonnet-4-20250514")
chain = prompt | model | StrOutputParser()

result = chain.invoke({"topic": "machine learning", "question": "What is overfitting?"})

LCEL: LangChain Expression Language

LCEL is LangChain's declarative composition syntax. Every component implements a Runnable interface with .invoke(), .stream(), .batch(), and .ainvoke() methods.

Advanced LCEL Patterns

from langchain_core.runnables import RunnablePassthrough, RunnableParallel

# Parallel execution: run two chains simultaneously
parallel = RunnableParallel(
    summary=summary_chain,
    keywords=keyword_chain
)

# Passthrough: keep original input alongside transformations
chain = RunnableParallel(
    context=retriever,
    question=RunnablePassthrough()
) | prompt | model | parser

# Branching: route based on input
from langchain_core.runnables import RunnableBranch
branch = RunnableBranch(
    (lambda x: "code" in x["topic"], code_chain),
    (lambda x: "math" in x["topic"], math_chain),
    default_chain  # fallback
)

LlamaIndex: Core Concepts

LlamaIndex (formerly GPT Index) is purpose-built for connecting LLMs to your data. While LangChain is a general orchestration toolkit, LlamaIndex specialises in retrieval-augmented generation (RAG).

Key Components

• Document Loaders — SimpleDirectoryReader, PDF, web, database, and API connectors via LlamaHub
• Node Parsers — Split documents into chunks (nodes) with configurable overlap and splitting strategies
• Indexes — VectorStoreIndex (semantic search), SummaryIndex (full scan), TreeIndex (hierarchical), KeywordTableIndex
• Query Engines — Combine retrieval and synthesis: retrieve relevant nodes, then generate an answer
• Response Synthesizers — compact, tree_summarize, refine strategies for assembling final answers

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.llms.anthropic import Anthropic

# Load documents, build index, query
documents = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine(
    llm=Anthropic(model="claude-sonnet-4-20250514")
)
response = query_engine.query("What are the key findings?")

Framework vs. Direct API Calls

The most important question is not "which framework?" but "do I need a framework at all?" Many production LLM applications use direct API calls and are better for it.

Multi-Provider Integration

Both frameworks support Claude, OpenAI, and open-source models through a unified interface. This makes it easy to benchmark providers or fall back between them.

# Claude
from langchain_anthropic import ChatAnthropic
llm = ChatAnthropic(model="claude-sonnet-4-20250514")

# OpenAI
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(model="gpt-4o")

# Local / Ollama
from langchain_ollama import ChatOllama
llm = ChatOllama(model="llama3")

# Claude
from llama_index.llms.anthropic import Anthropic
llm = Anthropic(model="claude-sonnet-4-20250514")

# OpenAI
from llama_index.llms.openai import OpenAI
llm = OpenAI(model="gpt-4o")

# Local / Ollama
from llama_index.llms.ollama import Ollama
llm = Ollama(model="llama3")

Both read API keys from environment variables (ANTHROPIC_API_KEY, OPENAI_API_KEY) automatically.

Lab & Homework