Theory

Reading time: ~20 minutes (~1500 words)  ·  Prerequisites: Lesson 24 (LoRA/QLoRA), OpenAI API key with fine-tuning access

1. What Is API-Based Fine-Tuning?

In Lesson 24, we explored fine-tuning with LoRA and QLoRA — techniques that require you to manage GPU hardware, training frameworks, and model weights yourself. API-based fine-tuning flips this model: you provide the data, and the provider handles everything else. You never see a GPU, never load model weights into memory, and never write a training loop.

The process is conceptually simple: you upload a file of training examples, specify a base model and a few hyperparameters, and the provider trains a customised version of that model for you. When training completes, you get a model identifier that you can use in regular API calls, exactly like you would use gpt-4o-mini or any other model — the only difference is that this one has been tuned on your data.

This approach is particularly appealing for teams that want the benefits of fine-tuning without the operational overhead of managing training infrastructure. The trade-off is less control: you cannot inspect gradients, modify the training loop, or apply advanced techniques like DPO or RLHF.

2. The JSONL Training Format

Every API fine-tuning provider has converged on a similar data format: JSONL (JSON Lines), where each line is a standalone JSON object representing one training example. The dominant structure is the chat-completions message format, which uses an array of messages with role and content fields.

A single training example looks like this:

{"messages": [
  {"role": "system", "content": "You are a technical writer. Write clear, concise documentation."},
  {"role": "user", "content": "Document the authenticate() function."},
  {"role": "assistant", "content": "## authenticate()\n\nValidates user credentials against the database.\n\n**Parameters:**\n- `username` (str): The user's login name\n- `password` (str): The plaintext password (hashed internally)\n\n**Returns:** A session token (str) on success, or raises `AuthError`."}
]}

The model learns to produce the assistant messages given the system and user context. It does not learn from the system or user messages directly — those serve as conditioning context. This means the quality of your assistant messages is the single most important factor in fine-tuning success.

Multi-Turn Examples

You can include multi-turn conversations by alternating user and assistant messages. This teaches the model conversational patterns:

{"messages": [
  {"role": "system", "content": "You are a helpful coding assistant."},
  {"role": "user", "content": "How do I read a CSV file in Python?"},
  {"role": "assistant", "content": "Use pandas: `import pandas as pd; df = pd.read_csv('file.csv')`"},
  {"role": "user", "content": "What if the file uses semicolons as separators?"},
  {"role": "assistant", "content": "Pass the sep parameter: `pd.read_csv('file.csv', sep=';')`"}
]}

3. Data Quality: The Make-or-Break Factor

The single biggest predictor of fine-tuning success is data quality. Fifty carefully crafted examples will consistently outperform five hundred sloppy ones. Here is a practical framework for evaluating your training data.

Consistency

Every assistant response should follow the same style, tone, and formatting conventions. If some examples use markdown headers and others use plain text, the model will randomly switch between them. If some are formal and others casual, the model will be inconsistently voiced. Before you start training, write a style guide for your assistant responses and verify that every example conforms to it.

Coverage

Your training data should cover the full range of inputs the model will encounter in production. If you fine-tune on only "happy path" examples, the model will struggle with edge cases, errors, and unusual inputs. Include examples of the model gracefully handling ambiguous questions, refusing inappropriate requests, and asking for clarification when needed.

Correctness

Every fact, code snippet, and instruction in your assistant messages must be accurate. The model will learn to reproduce errors just as readily as it learns to reproduce correct answers. Have domain experts review the training data before you upload it.

4. Validation and Automated Checks

Before uploading your data, run automated validation to catch structural issues. Here is a checklist of what to verify programmatically:

• JSON validity: Every line must parse as valid JSON. A single malformed line will cause the upload to fail.
• Schema compliance: Each object must have a messages key containing an array. Each message must have role (one of "system", "user", "assistant") and content (a non-empty string).
• Role ordering: Messages should follow a logical conversation flow. The last message should have role "assistant" (this is what the model learns to produce).
• Token counts: Each example must fit within the fine-tuning token limit (4,096 tokens for gpt-4o-mini, 16,384 for gpt-4o). Examples that exceed the limit are silently truncated, which corrupts your training signal.
• Duplicate detection: Exact duplicate examples waste training compute and can cause overfitting to those specific inputs.

OpenAI provides a data preparation notebook that performs these checks. We will build our own validation script in the lab.

5. Hyperparameters Explained

API fine-tuning exposes a small set of hyperparameters. Understanding what they do helps you make informed choices rather than guessing.

Number of Epochs (n_epochs)

An epoch is one complete pass through the training dataset. More epochs means the model sees each example more times, which generally improves learning — up to a point. Beyond that point, the model starts memorising the training data instead of generalising from it. This is called overfitting.

The right number of epochs depends on your dataset size. With fewer than 100 examples, 3–4 epochs is usually appropriate. With several hundred, 2–3 epochs suffices. With thousands, 1–2 epochs may be enough. Watch the training loss: if it drops to near zero, you are almost certainly overfitting.

Learning Rate Multiplier

This scales the provider's default learning rate. A value of 1.0 uses the default; higher values make larger weight updates per step (faster learning but potentially unstable), and lower values make smaller updates (slower but more stable). If your model is not learning enough after training, try increasing this. If it is producing nonsense or very repetitive outputs, try decreasing it.

Batch Size

The number of training examples processed together before updating model weights. Larger batches produce smoother gradient estimates but require more memory and can slow down the feedback loop. For most API fine-tuning use cases, the default (auto) is a reasonable starting point.

6. Monitoring and Evaluation

Once you submit a fine-tuning job, the provider trains the model asynchronously. You can poll the job status or, with OpenAI, stream training events. Key status transitions are: validating_files → queued → running → succeeded (or failed).

During training, the provider reports training loss (and validation loss, if you provided a validation file). Training loss should decrease over time. If you included a validation set, watch for the point where validation loss stops decreasing while training loss continues to drop — that is your overfitting threshold.

Post-Training Evaluation

After training completes, evaluate your fine-tuned model systematically:

1. Held-out test set: Run the fine-tuned model on examples it never saw during training. Compare outputs to expected answers.
2. Base model comparison: Run the same test prompts through the base model (with your original prompt engineering). Is the fine-tuned model actually better?
3. Regression testing: Test the fine-tuned model on general tasks outside your training domain. Fine-tuning can degrade general capabilities.
4. Human review: Have domain experts review a sample of outputs. Automated metrics often miss nuanced quality issues.

7. Cost Management and Versioning

Fine-tuning costs come in two parts: training cost (one-time, billed per token processed during training) and inference cost (ongoing, typically higher than the base model). For OpenAI, fine-tuned gpt-4o-mini inference costs roughly 2x the base model rate.

Despite the higher per-token inference cost, fine-tuning can reduce total costs if it allows you to eliminate long system prompts or few-shot examples from every request. A fine-tuned model that needs only a 50-token prompt instead of a 500-token prompt with examples saves 450 tokens per call — at scale, this adds up fast.

Model Versioning

Use the suffix parameter to give your fine-tuned models meaningful names (e.g., support-v1, support-v2). Keep a registry that maps each model version to its training data, hyperparameters, and evaluation results. When you deploy a new version, keep the previous one available for rollback. Delete old models you no longer need — storage is not free.

8. Key Takeaways

Further Reading

• OpenAI Fine-Tuning Guide — Official documentation covering the full API workflow.
• OpenAI Cookbook: Data Preparation — Validation and formatting utilities for JSONL data.
• OpenAI Fine-Tuning API Reference — Full parameter documentation.
• Google Vertex AI: Tune Models — Gemini fine-tuning documentation.
• Together AI Fine-Tuning — Fine-tuning open-source models via API.

Next lesson: Lesson 26 — Dataset Curation — building and maintaining high-quality training datasets at scale.