Lab Guide

Format: Jupyter notebook (lab.pdf)  ·  Time estimate: 45–60 minutes  ·  Prerequisites: Python 3.9+, tiktoken, anthropic, python-dotenv installed

Setup

Before starting, ensure your environment is ready:

# From the project root
pip install tiktoken anthropic python-dotenv

# Create a .env file in the project root with your API key
echo "ANTHROPIC_API_KEY=your_key_here" > .env

Verify setup by running the first notebook cell — it should import without errors and print your masked API key.

Exercise 1: Tokenisation with tiktoken

Goal

Understand how text is split into tokens and why this matters for cost and context window usage.

Steps

1. Import tiktoken and load the cl100k_base encoding (used by GPT-4-class models; we use it here as a practical proxy since Claude's tokeniser is not publicly released as a standalone library).
2. Encode a simple string (e.g., "Hello, world!") and print the token IDs.
3. Decode each token ID individually to see what text each token represents.
4. Count tokens for several different strings and compare the results.

Expected Output

String: "Hello, world!"
Token IDs: [9906, 11, 1917, 0]
Number of tokens: 4
Tokens: ['Hello', ',', ' world', '!']

String: "The quick brown fox"
Number of tokens: 5

String: "def calculate_fibonacci(n):"
Number of tokens: 9

String: "Привет, мир!"  (Russian: "Hello, world!")
Number of tokens: 8

Key Observations to Note

• The leading space before "world" is part of the token ' world' — this is a common source of confusion.
• Code tokenises less efficiently than natural language prose.
• Non-Latin scripts (Russian, Chinese, Arabic) typically require more tokens per character than English.
• Numbers split in surprising ways: "2024" might be ['202', '4'] or ['2', '024'] depending on context.

Exercise 2: Temperature Comparison

Goal

Observe how temperature affects model outputs for a creative task, and build intuition for when to use different temperature settings.

Steps

1. Define a creative prompt — something with no single "correct" answer, e.g.:

   "Write the opening sentence of a noir detective novel set in a cyberpunk city."

2. Call the Claude API twice with the same prompt:
   • Once with temperature=0
   • Once with temperature=1.0
3. Print both outputs side by side with clear labels.
4. Run the temperature=1.0 call three more times and observe the variation.

Expected Output

=== Temperature 0 ===
The rain fell like static through the neon-lit canyons of Neo Meridian,
and Detective Kai Voss stood at the edge of the crime scene hologram,
her augmented eyes cataloguing details that the uniformed officers had missed.

=== Temperature 1.0 (Run 1) ===
Chrome and shadows blurred together in the electric rain as I pushed
through the crowd outside the Neon Pagoda, a dead man's data chip
burning a hole in the pocket of my synth-leather jacket.

=== Temperature 1.0 (Run 2) ===
In a city where memory could be bought and sold, Detective Mara Chen
had learned the hard way that the only thing cheaper than a human life
was the digital ghost it left behind.

=== Temperature 1.0 (Run 3) ===
[output varies each run]

Key Observations to Note

• At temperature 0, repeated calls should produce identical (or near-identical) output.
• At temperature 1.0, each call explores different creative territory.
• Neither is "better" — they are appropriate for different tasks.

Exercise 3: Cost Calculator

Goal

Build a reusable function that estimates API cost given a prompt, expected output, and model name. This is a practical skill for production planning.

Steps

1. Define a dictionary of model pricing (cost per million tokens, separate input and output prices):

   MODEL_PRICING = {
       "claude-3-5-sonnet-20241022": {"input": 3.00, "output": 15.00},
       "claude-3-5-haiku-20241022":  {"input": 0.80, "output": 4.00},
       "claude-3-opus-20240229":     {"input": 15.00, "output": 75.00},
   }

2. Write a function estimate_cost(prompt: str, expected_output_tokens: int, model: str) -> dict that:
   • Counts input tokens using tiktoken
   • Calculates input cost and output cost separately
   • Returns a dict with input_tokens, output_tokens, input_cost_usd, output_cost_usd, total_cost_usd
3. Test it with a realistic scenario: a summarisation task with a 500-token article and 100-token expected summary.
4. Extend the function to calculate monthly cost given a calls_per_day parameter.

Expected Output

=== Cost Estimate ===
Model:          claude-3-5-sonnet-20241022
Input tokens:   487
Output tokens:  100 (estimated)
Input cost:     $0.001461
Output cost:    $0.001500
Total per call: $0.002961

=== Monthly Projection (100 calls/day) ===
Calls per month: 3000
Monthly cost:    $8.88

=== Model Comparison ===
Same workload across models:
  claude-3-opus-20240229     → $43.31/month
  claude-3-5-sonnet-20241022 → $8.88/month
  claude-3-5-haiku-20241022  → $1.99/month

Key Observations to Note

• Output tokens are significantly more expensive than input tokens on most models (5x on Sonnet, 5x on Opus).
• The cost difference between Haiku and Opus for the same task can be 20x or more.
• For high-volume applications, model selection has a larger impact on cost than prompt optimization.

Finishing Up

Once you have completed all three exercises:

1. Run all cells from top to bottom to verify everything executes without errors (Kernel → Restart & Run All).
2. Save the notebook.
3. Compare your implementation with lab_answer.pdf — focus on anything you did differently and whether your approach is equivalent or could be improved.

Self-Check

• ✓ Exercise 1: Can you explain why "Hello" and " Hello" produce different tokens?
• ✓ Exercise 1: Can you predict roughly how many tokens a 1000-word document would require?
• ✓ Exercise 2: Did your temperature=0 calls produce consistent output? Did temperature=1.0 vary?
• ✓ Exercise 2: Can you articulate when you would choose each temperature range?
• ✓ Exercise 3: Does your cost calculator return a dict with all five expected keys?
• ✓ Exercise 3: Can you explain why output tokens cost more than input tokens?