Operating an AI system: what is different here
LLMOps and production · Lesson 1 / 22
A normal service crashes; an AI system decays
A conventional backend has one convenient property: when it is broken, you can see it. 500s pile up in the logs, the queue grows, the endpoint stops responding. The on-call engineer wakes up to an alert, not to a customer email.
An AI system works differently. It almost never fails outright. It answers — on time, with status 200, in perfectly valid JSON — but it answers worse than it did a month ago. No technical chart will show that. So operations here do not start with uptime. They start with a question: what do we measure to know that the answers are still good?
Five differences the whole discipline grows from
- There is no single correct answer. You cannot compare a string against a reference, so you cannot write an ordinary assertion. Quality has to be described through properties of the answer, and through the share of cases where those properties hold.
- Behaviour changes without your release. The provider updated the model, changed defaults, altered how it handles long context. Your commit log is empty and the system is different.
- Every request costs money. Not amortised money for a server, but direct money proportional to traffic. A bad release here hits the invoice as well as the quality.
- The input is open text. A user can send anything, including instructions addressed to your model. Schema validation does not save you from that.
- Failure is soft. An ordinary bug returns a wrong result deterministically. Here the system is right in 92 percent of cases, and the entire job is noticing the slide to 85.
What this looks like on the team calendar
Normal service AI system ------------------------ ------------------------------------ release -> tests -> prod release -> eval run -> canary -> prod alert on 5xx alert on 5xx + refusal rate + spend incident = outage incident = outage OR quality drift rollback = code rollback = code, prompt, model, config on-call: fixes outages on-call: also decides on quality rollback
That is where the shape of this course comes from: versions and environments, continuous measurement of quality, observability and incident review, money and security. We do not cover the model, prompts, RAG or agents separately — neighbouring courses handle those. This one is about living with a finished system for months.
Where work on such a system begins
Before you build charts, answer three questions and write the answers down. First: which answer do we count as good — not 'useful', but described through checkable properties. Second: what share of bad answers do we tolerate — 5 percent, 15, none at all in money-related scenarios. Third: within what time must we learn that we crossed that share. Three answers take half a page and they determine everything else: the composition of the eval set, how often it runs, the alert thresholds, and who gets woken at night. Teams that skip this step spend six months arguing about charts, because the numbers have no agreed meaning behind them.
Cheat sheet
- An AI system degrades silently: the code is intact, quality creeps down.
- Behaviour changes without your release — from the provider side and from the data.
- Technical metrics are mandatory but not sufficient.
- Start by defining normal and the rollback threshold, not by picking tools.