One prompt, a pipeline or an agent: how to choose before you write any code

The freedom to choose a path costs money, latency and predictability

These three architectures solve overlapping classes of tasks, and the difference between them is who decides the order of steps. In a single prompt there is no order at all. In a pipeline you fix the order at development time. In an agent the model decides it at run time. Each move to the right adds capability and takes away predictability, so the decision should be deliberate rather than fashionable.

The line that separates them

There is exactly one question: is the graph of steps known before the run? If the sequence is the same for every input, you do not need an agent. You have handed the model something you could have written in code, and you are paying for extra calls and extra risk. If there are only a few branches and you can enumerate them, this is still a pipeline, just with conditions: let the model classify the input and let code choose the route. An agent is justified where the number of steps and their order depend on what turns up along the way, and the variants cannot be listed in advance.

task
 |-- answer follows from a single context?      -> one prompt
 |-- steps fixed, or fewer than 10 branches?    -> pipeline (code picks the route)
 |-- order depends on intermediate data
     and cannot be enumerated up front?         -> agent (model picks the route)

The price of the move, in numbers

It pays to estimate this before you implement anything. One prompt means one call, latency in seconds, and a cost you can predict up to the length of the input. A pipeline of four stages means four calls, latency that adds up, and a cost still known in advance to within a few percent. An agent means anywhere from 3 to 30 calls per task, a spread in cost of 5 to 10 times between runs on identical inputs, latency from seconds to minutes, and a share of runs that never reach a result which is rarely zero. Plan the budget against the 90th percentile of the step count, not the mean: it is the tail that decides the bill at the end of the month.

A hybrid almost always wins

In practice a pure agent is rarely what you need. The usual working shape is a pipeline with a short agent inside one of its stages, given two or three tools and a limit of 5 to 8 steps. Data collection, normalisation and writing the result stay deterministic, and free path-finding is granted only to the part that genuinely needs it. Such a system is debugged stage by stage, degrades partially rather than as a whole, and stays predictable in cost because the freedom is localised.

Cheat sheet

• The line is drawn by one question: is the graph of steps known before the run?
• Branches you can enumerate make a pipeline, not an agent.
• Budget an agent against the 90th percentile of the step count.
• The working shape is a pipeline with a short agent in a single stage.