Fine-tuning and customizing models

About this course

A course on fine-tuning language models as an engineering decision, not magic — when it beats a prompt or retrieval, and how to run it reproducibly. It covers training-data formats, hyperparameters, overfitting, baseline comparison, and operating model versions over time.

Five modules move from "fine-tune or not" through data and training to evaluation and operations. In "Data" you assemble a training-ready dataset with a task passport. In "Training" you run and document a full, reproducible training cycle for one model. In "Evaluation" you prepare a report comparing against a prompt baseline on the same data to decide on shipping.

For people with a real task and accumulated data or logs — without that, there's nothing to fine-tune. Requires Python and experience working with LLMs via API or prompts. This isn't a course on building neural nets from scratch — the focus is data, the training process, and whether it's worth doing.

What you'll learn

• Judge whether fine-tuning is worth it using a decision table: prompt, retrieval, fine-tuning
• Format a training example and decide what counts as the answer
• Collect and clean data: remove duplicates, contradictions, and test-set leakage
• Split data into sets before the first training run
• Tune hyperparameters and recognize overfitting from its symptoms
• Ensure reproducibility: data versions, config, and a run log
• Compare the fine-tuned model against a prompt baseline on the same set
• Assess cost, latency, and regressions after fine-tuning before shipping

What you'll end up with

• A training-ready dataset with its own task passport
• A reproducible report on the training cycle for one model
• An evaluation report on the fine-tuned model with a ship/no-ship call
• A quarter-long operations plan for the fine-tuned model (paid tiers include reviewer feedback and a certificate)

Not for you if you have no accumulated data for the task, or if a plain prompt already solves it without training.