Green tests and silent degradation
LLMOps and production · Lesson 2 / 22
Why ordinary tests miss a drop in quality
A unit test checks that a function returned the expected value. A generative answer has no expected value, so the team usually writes what can be written: a check that the answer is non-empty, that the JSON parses, that a field is present. Such tests stay green through any degradation of content.
What is actually checkable
- The format contract. Does the answer parse, are the required fields present, do the values fall inside the allowed range. This catches a broken integration; it does not catch nonsense in a correct format.
- Required and forbidden elements. An answer about returning a product must mention the deadline; a support answer must not contain an invented link to a section that does not exist.
- Invariants. If the request lacks data, the system must say so rather than make it up. You check that on deliberately incomplete inputs.
- Success rate over a set. Not pass or fail on a single example, but a percentage over a set of dozens of examples. It is the only quantity where comparing releases makes sense.
A threshold, not a binary result
Running the set gives you a number: 88 percent of answers meet the criteria, say. On its own it means nothing — what matters is the comparison with the previous run on the same set. A practical frame looks like this.
success rate drop under 2 pp -> noise, ship it drop of 2-5 pp -> investigate before rollout drop over 5 pp -> do not ship any drop on a critical slice -> do not ship regardless of the average model refusal rate doubles -> do not ship
Every team has its own numbers, but the threshold must be written down in advance. A threshold written after the release always turns out to be exactly loose enough to let that release through.
Spread between runs
The same set on the same prompt will give slightly different numbers. Before you react to a change, measure your own noise: run the set three times without changing a thing and look at the spread. If it is three points wide, you physically cannot tell a two-point improvement from chance — so either grow the set, or lower the temperature on evaluation runs, or stop discussing two-point victories.
The prompt test suite and the output contract were covered in detail in the prompting course; what matters here is different — this set stops being a one-off check before a release and becomes a regular measurement you take even without releases, because it is not only your code that can change.
Three kinds of false calm
Green status arrives from three places, and each of them deceives in its own way. Format checks are green because the model holds structure beautifully even while writing nonsense. The author's manual check is green because the author tests the examples the prompt was written for. User feedback is green because unhappy people stay silent: roughly one bad answer in twenty turns into a complaint, and the rest simply leave. None of those three signals replaces a regular run on a fixed set whose composition does not bend to fit the current version of the system.
Cheat sheet
- Ordinary tests catch the format, not the meaning.
- Compare the success rate over a set, not individual answers.
- Rollback thresholds are written before the release, not after it.
- Measure your own run-to-run noise before celebrating improvements.
Take your own AI feature (or any public task — for example, classifying support tickets). Assemble a set of 30 requests: 20 typical and 10 hard or edge cases. Describe the checkable criteria by which you count an answer as good. Run the set three times in a row without changing the prompt or the model, record the success rate for each run and the spread between them. Then state your thresholds: at what drop do you investigate, and at what drop do you refuse to ship.