Lesson 10 Building Agents
Theme:

Theory: Building Agents

Autonomous LLM systems that perceive, plan, and act using tools and memory.

Introduction

The term "agent" in the context of LLMs refers to a system where the language model autonomously decides which actions to take in a loop until a goal is achieved. This is a fundamental shift from the request-response pattern of standard LLM usage: instead of the human deciding what to do and the LLM helping with each step, the LLM itself decides the sequence of steps.

Agents have become the dominant paradigm for complex LLM applications. Code assistants, research tools, customer support systems, and data analysis pipelines all benefit from the agent pattern. The power comes from composability: a small set of well-defined tools, combined with the LLM's reasoning ability, can handle a wide range of tasks without writing task-specific code for each one.

What Makes an Agent

An LLM agent has four essential components:

The LLM serves as the decision-making engine. Given the current state (task + memory + latest observation), it decides: should I call a tool? Which one? With what arguments? Or should I produce a final answer? The quality of these decisions depends on the model's reasoning ability, the clarity of the tool descriptions, and the quality of the prompt.

ReAct

ReAct (Reasoning + Acting), introduced by Yao et al. (2023), is the most widely adopted pattern for LLM agents. It interleaves chain-of-thought reasoning with tool actions in a structured loop:

  1. Thought: The LLM articulates its reasoning — what it knows, what it needs, what it plans to do. This step is crucial because it makes the decision-making process explicit and debuggable.
  2. Action: Based on the thought, the LLM selects a tool and provides arguments. In Claude's tool-use API, this manifests as a tool_use content block.
  3. Observation: The tool result is fed back as a tool_result message. The LLM now has new information to reason about.

The loop continues until the LLM decides it has enough information to produce a final answer (indicated by stop_reason == "end_turn" with no tool calls) or until a maximum step limit is reached.

The elegance of ReAct is that it uses the LLM's existing capabilities — language understanding, reasoning, instruction following — to drive a general-purpose agent. No reinforcement learning, no custom training, no reward functions. The tool-use API handles the structured input/output, and the LLM's reasoning ability handles the planning.

def run_agent(task, tools, max_steps=10):
    messages = [{"role": "user", "content": task}]
    for step in range(max_steps):
        response = client.messages.create(
            model="claude-sonnet-4-20250514",
            max_tokens=1024,
            tools=tools,
            messages=messages,
        )
        # If the model is done, return its answer
        if response.stop_reason == "end_turn":
            return extract_text(response)
        # Execute tool calls and feed results back
        messages.append({"role": "assistant", "content": response.content})
        tool_results = execute_all_tools(response.content)
        messages.append({"role": "user", "content": tool_results})
    return "Max steps reached"

Memory

Memory determines what the agent can remember and for how long. Three types are commonly used:

In-Context Memory

The simplest and most common: the full conversation history (the messages list) serves as working memory. The agent can reference any previous tool call or result because it is still in the context window. Limitation: context windows are finite (typically 100k-200k tokens), and long agent runs can exhaust them.

External Key-Value Memory

A persistent dictionary or database where the agent can store and retrieve named values. Tools like save_note(key, content) and get_notes() provide this capability. This enables memory that persists across sessions and is not constrained by the context window.

Episodic Memory

Past interactions stored in a vector database. When starting a new session, the agent can retrieve relevant past conversations or decisions. This enables the agent to learn from experience — not by changing its weights, but by having access to a growing knowledge base of its own past actions and outcomes.

Planning

Complex tasks benefit from explicit planning before execution. Two key strategies:

Task Decomposition

Before acting, ask the LLM to break the task into sub-tasks. Each sub-task is simpler and more likely to succeed. The plan can be included in the system prompt or generated as the first step of the agent loop. This approach reduces errors because each step has a clearer objective.

Replanning

Plans often need to change when unexpected results occur. A robust agent includes checkpoints where it reassesses progress and potentially revises its plan. This can be triggered by: a tool returning an error, a search returning no results, or the agent detecting that its current approach is not converging on a solution.

System prompt pattern: "You are a research agent. Before starting, create a brief plan (3-5 steps). After each tool result, assess whether you are on track. If not, revise your plan."

Failure Modes

Agents fail in predictable ways that you can defend against:

Agent Architectures

Beyond the basic ReAct loop, several architectures have emerged:

Start with a single-agent architecture. Multi-agent systems add complexity that is only justified when tasks require genuinely different expertise or when parallel execution is needed.

Further Reading