Homework

Task 1: Tokenise Your Real-World Prompts

Goal: Build intuition for token counts and cost in your actual work.

Instructions

Identify 5 prompts you might use in your real work — or would use if you were using an LLM in your work today. These could be prompts for:

• Summarising documents
• Writing or reviewing code
• Answering questions about a codebase
• Drafting emails or reports
• Anything else relevant to your role

For each prompt:

1. Tokenise it using tiktoken (as in Exercise 1 of the lab).
2. Record:
   • The prompt (or a short description if it's long)
   • Token count
   • Estimated output tokens (make a reasonable guess for each use case)
3. Calculate the per-call cost using claude-3-5-sonnet-20241022.
4. Estimate monthly cost assuming you'd use each prompt 50 times per day.

Deliverable

A markdown table or printed output in this format:

Prompt description	Input tokens	Est. output tokens	Cost/call	Monthly cost (50/day)
Summarise meeting transcript	800	200	$0.0054	$8.10
...	...	...	...	...

Self-check

• ✓ You have 5 rows in your table.
• ✓ Token counts were computed programmatically (not guessed).
• ✓ Monthly costs use 30 days × 50 calls/day = 1500 calls/month.
• ✓ You can explain the biggest cost driver across your 5 prompts.

Task 2: Temperature Experiment

Goal: Develop your own intuitions about when temperature matters and how much.

Instructions

Choose a single creative writing prompt. Something open-ended where there is no single "right" answer — for example:

• "Describe what it feels like to debug code at 2am."
• "Write a two-sentence pitch for a product that doesn't exist yet."
• "Give an unusual analogy that explains what an API is."

Run this prompt through the Claude API at three temperature settings:

Temperature	Description
0	Deterministic / greedy
0.5	Balanced (We only tested 0 and 1.0 in the lab — 0.5 is for you to discover!)
1.0	Creative / sampled

For each temperature, collect three responses (nine responses total).

Deliverable

A written summary (~200–300 words) that addresses:

1. Consistency: How similar were the three outputs at temperature 0? At 1.0?
2. Quality: Did higher temperature produce better or worse outputs? Did it depend on the specific prompt?
3. Surprises: Was there anything unexpected in the outputs at any temperature?
4. Your rule of thumb: Based on your experiment, when would you use temperature 0, 0.5, and 1.0 in your own work?

Self-check

• ✓ You ran the exact same prompt at all three temperatures.
• ✓ You collected at least 3 responses per temperature level.
• ✓ Your summary is specific — it refers to actual differences you observed, not just "temperature 1 was more creative".
• ✓ You've stated a personal rule of thumb for temperature selection.

Task 3: The Context Window Trade-off

Goal: Reason clearly about why a large context window is not "free" to use.

Instructions

Modern LLMs advertise context windows up to 1 million tokens. This might seem to suggest that you can always stuff everything into a single prompt. This task asks you to think carefully about why that's not always true or wise.

Write exactly 100 words (±10) explaining why a 1M token context window doesn't mean you should always use it. Consider factors such as:

• Cost
• Latency
• "Lost in the middle" problem
• What the model actually pays attention to
• Prompt design discipline

Your answer should be written for a non-technical stakeholder — clear, concrete, and jargon-free where possible.

Deliverable

A 100-word paragraph saved in a text file or included in your homework submission.

Self-check

• ✓ Your answer is 90–110 words.
• ✓ You mention at least three distinct reasons (not just cost three different ways).
• ✓ A non-technical reader could understand your answer.
• ✓ You don't just list bullet points — it reads as a coherent, connected explanation.