Homework — Three Extensions of the Research Assistant

Task 1 — Add a save_note tool so the agent learns between runs

Add a sixth tool that lets the agent write back into PKB — the in-process dictionary that pkb_search and pkb_fetch read from. The signature:

@tool
def save_note(note_id: str, title: str, text: str) -> str:
    """Persist a note into the personal knowledge base under `note_id`.
    Use this after summarising a web source so the next run can hit PKB first."""
    PKB[note_id] = {"title": title, "text": text}
    return f"saved {note_id}"

Register save_note in the agent’s tools=[...] list and add a line to the system prompt nudging the model to save useful findings, e.g.: “After you summarise a web source, save the summary back to the knowledge base using save_note so future runs are cheaper.”

Run the agent on the same question twice in the same Python process (a tiny driver script with two agent.run(...) calls works; you cannot run python agent.py twice because PKB resets between processes):

from agent import agent

q = "How do tool-heavy agents differ from chatbots?"
print("--- run 1 ---"); agent.run(q)
print("--- run 2 ---"); agent.run(q)

✓ Self-check: on run 2, pkb_search returns hits that run 1 didn’t have (the notes the agent saved on run 1). The agent should call web_search fewer times — or not at all — the second time. result.usage.total_cost_usd for run 2 should be visibly lower than run 1.

Task 2 — Write an evaluate_report function and run it on three questions

Outside the agent (top-level helper in your driver script), write:

def evaluate_report(report: dict, cost_usd: float) -> dict:
    """Return four basic quality metrics for a research-assistant report."""
    ...

The returned dict must contain exactly these four keys:

• has_intro — bool, True iff report["intro"] is at least 20 characters long.
• bullet_count — int, len(report["bullets"]).
• source_count — int, len(report["sources"]).
• cost_per_bullet_usd — float, cost_usd / bullet_count, or None if bullet_count == 0.

Then drive the agent on three different questions and print the eval per question:

questions = [
    "How do tool-heavy agents differ from chatbots?",
    "What is the Claude Agent SDK’s `@tool` decorator for?",
    "Why does the LLM need docstrings on its tools?",
]
for q in questions:
    result = agent.run(q)
    report = result.final_output           # the dict write_report produced
    cost   = result.usage.total_cost_usd
    print(q, "→", evaluate_report(report, cost))

✓ Self-check: all four metrics are populated for all three questions. Reasonable values: has_intro is True, bullet_count in the 2–5 range, source_count ≥ 1, cost_per_bullet_usd under $0.02. If cost_per_bullet_usd is None for any question, the agent skipped write_report — check your system prompt.

This is the seed of the cross-framework scorecard we’ll build in Module 6: the same four metrics will let you compare this Claude Agent SDK implementation against the LangGraph, OpenAI SDK, and CrewAI re-implementations of the same agent.

Task 3 — Make write_report reject unsourced bullets

Modify the write_report tool so that every bullet must be grounded — appearing (case-insensitive substring match) in at least one of the sources strings or in the original user question. If any bullet fails the check, raise ValueError("bullet not backed by a source").

def is_grounded(bullets: list[str], sources: list[str], question: str) -> list[bool]:
    """Debug helper: per-bullet boolean of whether the bullet is grounded."""
    haystack = " ".join(sources + [question]).lower()
    return [b.lower() in haystack for b in bullets]

@tool
def write_report(intro: str, bullets: list[str], sources: list[str]) -> dict:
    """Produce the final report. Bullets must be grounded in sources or the question."""
    grounded = is_grounded(bullets, sources, _current_question)
    if not all(grounded):
        raise ValueError("bullet not backed by a source")
    return {"intro": intro, "bullets": bullets, "sources": sources}

You’ll need to capture the original question into a module-level _current_question string before calling agent.run(question), since tools don’t receive the original prompt directly. (The Solutions page shows one clean way to do that.)

Run the agent on the same three questions from Task 2.

✓ Self-check: at least one of the three questions causes write_report to raise ValueError on the first attempt. The SDK surfaces the error to the model as is_error: true, and the model retries with bullets that do appear in its sources. Final reports are grounded; you can verify by running is_grounded(report["bullets"], report["sources"], question) — all True.