Prompting, document search, fine-tuning: a decision table

Three tools with different failure points

In the prompt engineering course you pinned down an output contract and ran a test suite; in the document search course you fed retrieved fragments into the context. Fine-tuning arrives third and replaces neither of them. The right question is not which one is better, but which failure you are repairing. A prompt repairs a misunderstood task. Search repairs missing data. Fine-tuning repairs an unstable form when the task is already understood and the data is already there.

The decision table

symptom                                    tool
──────────────────────────────────────────────────────────
the model did not grasp what is wanted     prompt: output contract
answer right in form, wrong in substance   document search
the format drifts from request to request,
the instruction is already 1500+ tokens    fine-tuning
numbers and aggregates are needed          a query to the database
the task changes every two weeks           prompt (flexibility wins)
the answer must cite a source              document search
──────────────────────────────────────────────────────────

An order of work, not a hierarchy

The order is this: push the prompt to its ceiling while measuring it on a test suite, then add search if the errors are factual, and only then consider fine-tuning. The reason is not ideology but the cost of change. You ship a prompt edit in a minute, a change to the document set in an hour, a new version of the weights in days, counting data collection, training, evaluation and rollout. Starting with the most expensive iteration cycle means paying that price on every hypothesis.

There is a practical bonus in this order too: a prompt pushed to its ceiling becomes a generator of training data. Its outputs, checked and corrected by a human, are ready-made training examples. A team that skipped the prompt stage later has to build its labeled dataset from nothing.

Combination instead of choice

The working architecture is most often hybrid: search is responsible for content, the fine-tuned model for form. Take a support assistant where policy fragments are fed into the context, while fine-tuning makes sure the answer always opens with the resolution, states the deadline, and carries no apologetic filler. Neither mechanism solves the other one's problem, and replacing search with fine-tuning here makes no sense.

Signs that the prompt has genuinely stalled

• The instruction has sprawled. Fifteen hundred to two thousand tokens of rules you are afraid to touch, because editing one clause breaks another.
• Instability on repeats. The same input at low temperature produces a different answer structure in a noticeable share of runs.
• Examples in the prompt stop helping. Adding examples gives a lift on the first three to five and then hits a plateau.
• Volume justifies the cycle. The task lives for months and handles thousands of requests a day, not fifty.

Cheat sheet

• The prompt repairs misunderstanding, search repairs missing data, fine-tuning repairs unstable form.
• The order is dictated by the cost of iteration, not by fashion.
• A prompt pushed to its ceiling yields draft training examples.
• The hybrid of search for facts and fine-tuning for form is the standard working scheme.