What fine-tuning actually changes in a model

Fine-tuning and customizing models · Lesson 1 / 20

A distribution shift, not a knowledge upload

Fine-tuning looks like showing the model your materials, but mechanically it is something else entirely. The model predicts the next token and emits a probability distribution over the vocabulary. Training on your input-output pairs shifts that distribution so that the tokens of your answers become more likely in your contexts. There is no separate cell inside the model where a fact gets filed away: the change is smeared across the weights and shows up as a tendency to answer in a particular way.

Everything else follows from this. Form transfers well: answer length, structure, section order, terminology, tone, the absence of a preamble. It transfers because it is a statistical habit repeated across hundreds of training examples. A fact transfers poorly, because it appears once and has to be reproduced exactly. After fine-tuning the model cannot separate what it learned from what it filled in, and it cannot cite a source, because it has no source.

What transfers and what does not

Output format. A stable answer structure, a fixed set of fields, a fixed order of blocks. This is the most reliable transfer of all.
Style and register. A dry procedural tone instead of an expansive one, formal address, no apologies.
Task type. Labeling a ticket with your taxonomy, extracting entities by your rules, rewriting text to your standard.
Behavioral boundaries. When to answer, when to ask a clarifying question, when to refuse and with which wording.
Facts and numbers. These transfer unreliably: prices, dates, names and clause numbers come back approximate and confident.

A one-minute suitability check

Take the answer you want and ask a simple question: if a person knew only the request and your rulebook, but had no access to the database, could they write this answer? If yes, the task is about form and fine-tuning fits. If producing the answer requires looking up a value in a table or reading a document, this is a knowledge task, and the way to solve it is to put the data into the context.

request: "Refund the money for order 4471"
──────────────────────────────────────────────
taken from context:  order status, amount, refund window
given by fine-tuning: order of answer blocks, tone, no extra
                      apologies, mandatory mention of the window
──────────────────────────────────────────────
fine-tuning with no context -> plausible amount, invented window

Why this is not an argument about words

The form-versus-facts distinction sets the budget of the project. Form is stable: you collect five hundred training examples, train once, and it works for months. Facts move: every price change demands a fresh cycle of data collection, training, evaluation and rollout. Teams that start fine-tuning in order to inject knowledge usually discover this on the third retraining cycle, a quarter of a year in.

Insight. Fine-tuning compresses the prompt. What you used to spell out in two thousand tokens of instructions on every single request costs zero tokens after training, because the rule has moved into the weights. At high call volume this is the main money effect, not the gain in quality.

Common mistake. Expecting the model to memorize the documentation. Train it on documentation text and it will start speaking in that style and confidently naming sections that do not exist. You get the style; you do not get the accuracy.

Pro tip. Before you collect any data, write out ten real answers you want and underline everything in them that cannot be derived from the request. If a lot is underlined, the project will fail in this framing, and it is much better to learn that before labeling starts.

Cheat sheet

Fine-tuning shifts a probability distribution; it does not add memory.
Form, tone, structure and task type transfer; facts do not.
The check: can the answer be written knowing only the rules and the request.
The main saving from fine-tuning is a short prompt, not a leap in quality.

1. Why do facts transfer into the weights worse than the answer format does?

2. What is the one-minute check that a task suits fine-tuning?

3. Which money effect of fine-tuning shows up at high call volume?

🔒 Answer the question correctly to move on to the next lesson.