Homework — Lesson 02

Three take-home tasks to deepen your understanding of the Anthropic API. Submit a single .py file (or a short notebook) for each task.

Task 1

Conversation History Manager

Build a CLI chatbot that maintains up to 10 turns of conversation history. When the conversation reaches 10 turns, use Claude to summarise the entire conversation into a concise system-prompt-style paragraph, then start fresh with that summary as the new context. This prevents the context window from growing indefinitely while preserving important information.

Requirements

The chatbot reads from stdin and responds to the terminal (no web UI needed).
Maintain a history list of {"role": ..., "content": ...} dicts.
After every assistant reply, check if len(history) >= 20 (10 user + 10 assistant turns).
When the limit is hit, make a separate API call asking Claude to summarise the conversation in 2–3 sentences as if it were a system prompt (e.g., "The user's name is Alex, they are building a Python web app and have asked about routing and authentication.").
Reset history = [] and start the next turn with the summary injected as the system parameter.
Print a visible separator when summarisation occurs so the user can see the context was reset.

def summarise_history(history: list[dict]) -> str:
    """Ask Claude to summarise the conversation into a system-prompt paragraph."""
    summary_prompt = (
        "Summarise the following conversation in 2-3 sentences as a system-prompt context block. "
        "Focus on key facts, user preferences, and topics discussed.\n\n"
        + "\n".join(f"{m['role'].upper()}: {m['content']}" for m in history)
    )
    response = client.messages.create(
        model="claude-3-5-haiku-20241022",
        max_tokens=256,
        messages=[{"role": "user", "content": summary_prompt}]
    )
    return response.content[0].text

Self-check:

Bot correctly summarises at turn 10 and prints a separator message
After summarisation, len(history) == 0 (fresh start)
Claude still answers follow-up questions using information from the summary
History never exceeds 20 items (10 user + 10 assistant) at any point

Task 2

Model Switcher with Benchmarking

Add a --model flag to a script using Python's argparse module. Support haiku, sonnet, and opus as shorthand aliases for the full model IDs. Run the same 3 prompts against both Haiku and Sonnet, measure response time and token usage, and write a short comparison note.

Requirements

Use argparse to accept a --model argument with choices ["haiku", "sonnet", "opus"].
Map shorthand names to full model IDs using a dictionary.
Use these 3 test prompts:
1. "Classify the sentiment of this review as positive, negative, or neutral: 'The product arrived damaged and customer service was unhelpful.'"
2. "Write a Python function that checks whether a string is a palindrome."
3. "Explain the concept of gradient descent in machine learning in 3 sentences."
Measure elapsed time for each call using time.time().
Print a table: Prompt # | Model | Time (s) | Input Tokens | Output Tokens.
After running, write a 100-word comparison note in a comment at the bottom of your script.

import argparse
import time
import anthropic
from dotenv import load_dotenv

load_dotenv()
client = anthropic.Anthropic()

MODEL_MAP = {
    "haiku":  "claude-3-5-haiku-20241022",
    "sonnet": "claude-3-5-sonnet-20241022",
    "opus":   "claude-3-opus-20240229",
}

def parse_args():
    parser = argparse.ArgumentParser(description="Benchmark Claude models")
    parser.add_argument("--model", choices=["haiku", "sonnet", "opus"],
                        default="haiku", help="Model shorthand to use")
    return parser.parse_args()

# ... implement benchmark_prompt() and main() ...

Tip: You can run the script twice from the terminal — once with --model haiku and once with --model sonnet — and redirect the output to a file for easy comparison: python benchmark.py --model haiku > haiku_results.txt

Self-check:

python benchmark.py --model haiku runs without errors
python benchmark.py --model sonnet produces visibly different quality on prompt 2 (code)
Times are measured correctly with time.time() wrapping the API call
Your 100-word comparison note addresses quality, latency, and token usage differences

Task 3

Retry with Exponential Backoff

Implement call_with_retry(prompt, max_retries=3) that catches anthropic.RateLimitError and retries with exponential backoff starting at 1 second: wait 1s after the first failure, 2s after the second, 4s after the third. Log each retry attempt clearly. Then write a unit test using unittest.mock.patch to verify the retry logic without actually hitting the API.

Requirements

Catch anthropic.RateLimitError specifically — do not catch all exceptions.
Wait times: 2 ** attempt seconds (attempt 0 → 1s, attempt 1 → 2s, attempt 2 → 4s).
Print a log message for each retry: f"[Retry {attempt+1}/{max_retries}] Rate limited. Waiting {wait}s..."
If all retries are exhausted, raise a RuntimeError with a descriptive message.
Write a test that patches client.messages.create to raise RateLimitError twice then succeed on the third attempt. Assert the result is correct and that create was called exactly 3 times.

import time
import anthropic
from unittest.mock import patch, MagicMock

def call_with_retry(client, prompt: str, max_retries: int = 3) -> str:
    """Call the API with exponential backoff on rate limit errors."""
    for attempt in range(max_retries):
        try:
            response = client.messages.create(
                model="claude-3-5-haiku-20241022",
                max_tokens=256,
                messages=[{"role": "user", "content": prompt}]
            )
            return response.content[0].text
        except anthropic.RateLimitError:
            wait = 2 ** attempt
            print(f"[Retry {attempt + 1}/{max_retries}] Rate limited. Waiting {wait}s...")
            time.sleep(wait)
    raise RuntimeError(f"API call failed after {max_retries} retries")


# --- Unit test ---
def test_retry_logic():
    """Verify call_with_retry retries on RateLimitError and succeeds on 3rd attempt."""
    mock_client = MagicMock()

    # Simulate: fail, fail, succeed
    success_response = MagicMock()
    success_response.content = [MagicMock(text="Hello!")]

    mock_client.messages.create.side_effect = [
        anthropic.RateLimitError("rate limited", response=MagicMock(), body={}),
        anthropic.RateLimitError("rate limited", response=MagicMock(), body={}),
        success_response,
    ]

    with patch("time.sleep"):  # don't actually sleep during tests
        result = call_with_retry(mock_client, "Say hello.")

    assert result == "Hello!"
    assert mock_client.messages.create.call_count == 3
    print("test_retry_logic PASSED")

test_retry_logic()

Why mock instead of testing against the real API? A unit test that makes real API calls is slow, flaky (depends on network), and costs money. By mocking the client you can test the retry logic deterministically in milliseconds — and you can simulate rare conditions (like 3 consecutive rate limit errors) that would be hard to trigger in a real environment.

Self-check:

call_with_retry handles 0, 1, 2, and 3 failures correctly
After max_retries failures, a RuntimeError is raised (not silenced)
test_retry_logic() prints "PASSED" without hitting the real API
The mock asserts call_count == 3 (two failures + one success)
time.sleep is patched in the test so it runs instantly