Solutions — Three Triage-Agent Extensions

Lesson 08 · Building Your Own Agents · worked answers with commentary

Below is one clean answer per homework task, together with the why-this-not-that commentary that turns a passing solution into an idiomatic one. All three solutions extend agents/lesson08-handoff-agent/agent.py from the lab — copy the snippet into the file (or a sibling agent_homework.py), restart the process, and the new behaviour shows up on the next Runner.run_sync(...).

How to read the solutions

Each section opens with the full code, then a common mistake callout, then the broader lesson. The three extensions stack: by the end you have a triage agent that is guarded against runaway refunds, capable of cyclic handoffs, and measurable via a route-accuracy benchmark — the exact dimensions Module 6's cross-framework scorecard scores on.

Solution to Task 1 — an output guardrail that caps refund amounts

The guardrail inspects the Resolution's actions_taken for refund lines matching "refund of $X.XX" and trips on anything above $1000. Drop this into agent.py alongside the other imports and the billing agent definition:

import re
from agents import output_guardrail, GuardrailFunctionOutput


REFUND_AMOUNT_RE = re.compile(r"refund of \$([0-9,]+\.[0-9]{2})")


@output_guardrail
async def cap_refunds(ctx, agent, output):
    actions = getattr(output, "actions_taken", [])
    big = []
    for a in actions:
        m = REFUND_AMOUNT_RE.search(a)
        if m and float(m.group(1).replace(",", "")) > 1000:
            big.append(a)
    return GuardrailFunctionOutput(
        output_info={"oversized_refunds": big},
        tripwire_triggered=bool(big),
    )


billing = Agent(
    name="Billing",
    instructions=BILLING_INSTRUCTIONS,
    model="gpt-4o-mini",
    tools=[lookup_invoice, issue_refund],
    output_type=Resolution,
    output_guardrails=[cap_refunds],
)

Trigger. Run a ticket that demands a large refund:

try:
    run_support_agent(
        "Please refund $5000 for last month's outage against invoice INV-42."
    )
except Exception as e:
    print(type(e).__name__, getattr(e, "output_info", None))

Expected:

OutputGuardrailTripwireTriggered {'oversized_refunds': ['issued refund of $5,000.00 against INV-42']}

What changed. The billing agent still drafts a complete Resolution — the guardrail runs after the agent produces its typed output but before the SDK returns it to the caller. The structured output_info dict survives on the raised OutputGuardrailTripwireTriggered exception, so your handler can route the ticket to a manual-review queue with full context instead of just “blocked”. In production you'd swap the print for an alert + queue insert.

Common mistake — placing `cap_refunds` on triage

Output guardrails can only see the agent's final output. Triage never produces a final output — it hands off. Bolt the guardrail onto triage and it simply never fires, no matter how large the refund downstream. Output guardrails belong on the agent that actually answers, which for refunds is billing only. If you want a refund-cap that applies to multiple specialists, add the same output_guardrails=[cap_refunds] to each one — they're cheap.

Why output guardrails for money matter. The LLM may be correct most of the time, but the cost-of-failure on a runaway refund is asymmetric: one $5000 mistake dwarfs the savings from a thousand correct $50 resolutions. Output guardrails are the cheap insurance — one decorator plus a regex turns a quiet correctness bug into a loud, catchable exception with structured diagnostics.

Solution to Task 2 — cyclic handoff (Technical hands back to Triage)

The OpenAI Agents SDK supports cycles, but the handoffs=[...] argument takes concrete Agent instances at construction time — not forward references. The cleanest pattern is to construct specialists first (without handoffs), build triage with all three specialists, then assign the back-handoff on technical via attribute assignment. The full wiring:

# Step 1 — specialists, no handoffs yet.
billing = Agent(
    name="Billing",
    instructions=BILLING_INSTRUCTIONS,
    model="gpt-4o-mini",
    tools=[lookup_invoice, issue_refund],
    output_type=Resolution,
)

technical = Agent(
    name="Technical",
    instructions=TECHNICAL_INSTRUCTIONS,  # see updated text below
    model="gpt-4o-mini",
    tools=[check_service_status, open_incident],
    output_type=Resolution,
)

account = Agent(
    name="Account",
    instructions=ACCOUNT_INSTRUCTIONS,
    model="gpt-4o-mini",
    tools=[reset_password, unlock_account],
    output_type=Resolution,
)

# Step 2 — triage with all 3 specialists.
triage = Agent(
    name="Triage",
    instructions=TRIAGE_INSTRUCTIONS,
    model="gpt-4o-mini",
    handoffs=[billing, technical, account],
    input_guardrails=[block_pii],
)

# Step 3 — add the back-handoff on technical AFTER triage exists.
technical.handoffs = [triage]

Append one sentence to TECHNICAL_INSTRUCTIONS so the model knows when to use the back-handoff:

TECHNICAL_INSTRUCTIONS = TECHNICAL_INSTRUCTIONS + (
    "\nIf the user also has a billing concern, hand back to Triage so Billing "
    "can pick it up."
)

Trigger. Run a mixed ticket:

out = run_support_agent(
    "Service is down AND I want a refund for the downtime."
)
print(out["last_agent"])           # Billing
print([s.name for s in out["trace"]])  # Triage, Technical, Triage, Billing

Expected trace: Triage → Technical (calls check_service_status and open_incident) → Triage → Billing (calls lookup_invoice and issue_refund). The final result.last_agent.name is "Billing".

Common mistake — `handoffs=[triage, billing]` on technical

Tempting shortcut: let technical route directly to billing. It works, but it bypasses triage entirely — including the block_pii input guardrail on triage, the per-ticket logging you'll add later, and any future triage-only policy. Always route handoffs through the agent whose instructions and guardrails you want to enforce. The slight extra hop (Technical → Triage → Billing) costs ~$0.0005 per ticket; the discipline saves you a class of subtle bypass bugs.

Why cyclic handoffs are useful but dangerous. Cycles unlock genuine multi-step routing — the user's real intent often spans two categories, and a flat one-hop router can't express that. But a poorly-instructed cycle can loop indefinitely (Technical → Triage → Technical → Triage …). The SDK does not ship a default max-handoff cap; for production set max_turns= on Runner.run(...) as a hard backstop, and write the instructions so each handoff narrows the remaining work rather than re-opening it.

Solution to Task 3 — a `route_accuracy` evaluator

Ten labelled tickets covering all three specialists, plus a driver that prints per-ticket pass/fail and the overall accuracy. Drop this into the bottom of agent.py (replacing the existing if __name__ == "__main__" block) or into a sibling eval.py:

BENCHMARK = {
    "My invoice is wrong.": "billing",
    "March bill is double.": "billing",
    "Cancel auto-renewal.": "billing",
    "The site is down.": "technical",
    "Checkout returns 500 errors.": "technical",
    "API latency spiked.": "technical",
    "I can't log in.": "account",
    "Lost my password.": "account",
    "Account is locked.": "account",
    "My email address changed.": "account",
}


def route_accuracy():
    correct = 0
    for ticket, expected in BENCHMARK.items():
        out = run_support_agent(ticket)
        actual = out["last_agent"].lower()
        ok = actual == expected
        correct += ok
        print(
            f"[{'PASS' if ok else 'FAIL'}] "
            f"{actual:>10s} (expected {expected:>10s}) | {ticket}"
        )
    accuracy = correct / len(BENCHMARK)
    print(f"\noverall accuracy: {accuracy:.1%} ({correct}/{len(BENCHMARK)})")
    return accuracy


if __name__ == "__main__":
    route_accuracy()

Expected output (one representative run). On gpt-4o-mini with the stock TRIAGE_INSTRUCTIONS you typically see 90–100% accuracy. A representative run:

[PASS]    billing (expected    billing) | My invoice is wrong.
[PASS]    billing (expected    billing) | March bill is double.
[PASS]    billing (expected    billing) | Cancel auto-renewal.
[PASS]  technical (expected  technical) | The site is down.
[PASS]  technical (expected  technical) | Checkout returns 500 errors.
[PASS]  technical (expected  technical) | API latency spiked.
[PASS]    account (expected    account) | I can't log in.
[PASS]    account (expected    account) | Lost my password.
[PASS]    account (expected    account) | Account is locked.
[FAIL]    billing (expected    account) | My email address changed.

overall accuracy: 90.0% (9/10)

Bonus — fixing the failure. The miss above is the borderline case: “My email address changed” reads as a billing-profile update to the model. Tighten TRIAGE_INSTRUCTIONS with an explicit example:

TRIAGE_INSTRUCTIONS = TRIAGE_INSTRUCTIONS + (
    "\n\nExamples:\n"
    "- 'My email address changed.' → Account (it's an identity update, "
    "not a payment-method change)\n"
    "- 'My credit card on file expired.' → Billing\n"
    "- 'I can't reach 2FA codes.' → Account\n"
)

Re-run; accuracy should climb to 100%. The pattern — measure first, fix the worst category, re-measure — is exactly how Module 6's cross-framework scorecard is generated.

Common mistake — single-run measurement and over-fitting

Reporting accuracy from a single benchmark pass and tightening instructions to chase that single number. The model is non-deterministic — the same ticket can route differently across runs (especially borderline cases). Real route accuracy needs ≥3 runs per ticket; average across runs, not a single-run number. Set temperature=0 if you want lower variance, but it does not eliminate it — only reduces it. The Module 6 scorecard reports mean ± standard deviation across 5 runs.

Why this matters for Module 6. “Route accuracy” is one of the dimensions on the cross-framework scorecard. The OpenAI SDK's strength is concentrated here — because handoffs are first-class primitives (not hand-rolled router functions), the same instructions yield higher accuracy than the equivalent Claude-SDK or LangGraph rewrite. The numbers will speak in Lesson 11.

What you’ve built

Across three small extensions you have made the triage agent guarded (refund cap via output guardrail), flexible (cyclic handoff that handles mixed-intent tickets), and measurable (a 10-ticket benchmark with per-ticket pass/fail and an overall accuracy number). Those are the three knobs you will reach for in every real triage-and-handoff agent:

Output guardrails on the specialists that handle money or destructive operations — not on triage, because triage doesn't produce a final output.
Cyclic handoffs routed through triage — so the triage-only policies (PII guardrail, logging, future invariants) keep firing on every routing decision.
A small, labelled benchmark — ten tickets is plenty to catch the obvious regressions and to score against the next framework in Module 6.

Closing tip: You've now built the canonical OpenAI Agents SDK use case. Module 5 introduces CrewAI, where you'll meet a fourth shape: agents that don't hand off but collaborate — a planner, a worker, a reviewer working together on the same brief.

← Back to homework