Pick retrieval or a workflow before you reach for an agent

A product team usually asks for an agent about a week after the first demo. The demo answered questions well, someone asked what happens when it needs to look up an order or send an email, and the conversation jumps straight to autonomy. That jump skips two cheaper designs that cover most of what the team actually wants. Retrieval fixes a model that does not know your facts. A workflow fixes a model that needs to do your steps in your order. An agent, meaning a system that picks its own next step at runtime, is for the case where the sequence genuinely cannot be written down in advance. Choosing wrong is not only an over-engineering problem. It moves failure from something you can test into something you can only watch happen.

Who chooses the next step

The useful line between these shapes is not how capable the model is. It is who decides what happens next. Anthropic's engineering guidance draws the line by control flow: workflows keep LLMs and tools inside predefined code paths, while agents choose their own sequence and tool use at runtime (Building effective agents). Retrieval sits below both. It changes what the model knows for a single call without changing who is driving. The original formulation paired a pretrained generator with a dense index so generation could use both learned parameters and retrieved documents (Lewis et al., 2020); production RAG is that idea pointed at your own content.

Each step away from predefined code paths buys flexibility and sells back predictability. That is the entire trade. The same guidance notes that agents add latency and cost, while many applications can be solved by a single model call improved with retrieval and in-context examples.

Retrieval answers knowledge questions, not action questions

Start from the complaint. If users say "it made that up," "it doesn't know our refund policy," or "it's quoting last year's pricing," you have a knowledge problem and retrieval is the whole fix. If they say "it should have opened a ticket," you have an action problem and retrieval will not touch it.

Retrieval also earns its place because its failures are legible. When an answer is wrong, you can look at what was retrieved and see whether the index was stale, the chunk boundary cut the answer in half, or the ranking buried the right passage in position nine. That is a debuggable system.

The assistant on the Alfcode site is deliberately this shape: one model call with a server-authoritative system prompt, no tools, and a fixed reply pointing to a person whenever the model is unavailable. Any system field sent by the browser is ignored. It can explain how we work; it cannot do anything on a visitor's behalf. Nothing about that job needs autonomy, and adding autonomy would only add ways to fail.

Workflows carry most of what ships

Most production AI is a known sequence with judgment in one or two slots: classify the incoming message, route it, draft a reply, validate the draft against rules, commit. The model does the part that needs language. Code does the part that needs to be identical every time.

What you get back is ordinary engineering. You can test the path, retry one step without rerunning the others, log exactly what ran and why, cap spend per execution, and put an approval in front of the single action that is hard to reverse. Event triggers, retries, and audit records are well understood, and the model does not get to reinvent them per run.

The editorial pipeline behind this journal works that way. A timer starts a read-only research pass once a day. The model researches one queued topic and has to verify at least one external primary source. Publication is not the model's call: a deterministic gate checks the result, and anything that fails stays a draft for a person, invisible to readers until someone approves it. Manual runs go through the same lock, sandbox, and gate. The model has real latitude inside a step; the workflow owns the order of steps and the irreversible action at the end.

What actually earns an agent

Three conditions, and you want all three present.

The branching is genuinely unenumerable. Debugging a failure across systems, researching a question where the next query depends on what the last one returned, reconciling records that disagree in ways nobody can list ahead of time. Not "there are many cases," but cases you cannot write down.

There is a cheap, reliable check on the output. Code that compiles and passes tests, a ledger that balances, a document that validates against a schema. An agent without a verifier produces confident work with no way to grade it, and you find out weeks later.

The blast radius is bounded by something other than the model's judgment: scoped credentials, spend limits, reversible actions, an approval before anything that touches money or a customer.

Missing the second condition gives you a system you cannot evaluate. Missing the third gives you one that can do damage faster than anyone notices. Missing the first means you paid agent prices for workflow behavior.

Evals, budgets, and the exit to a person

You do not settle this on a whiteboard. Build the evaluation set first: thirty to fifty real cases with expected outcomes, taken from actual traffic or actual tickets. Then run the ladder. Does a single call with retrieval pass? If not, does a fixed workflow? Only then consider autonomy. That set is also your regression signal afterwards, which is the difference between knowing a prompt change made things worse and hearing it from a customer.

Cost and latency follow the same order and belong in the decision, not in a surprise invoice two months later. A retrieval answer is one call with a predictable tail latency. A workflow's cost is countable because its steps are countable. An agent's cost is a function of choices it makes at runtime, so it needs explicit step and budget ceilings, plus routing so cheap steps do not run on the expensive model.

Design the exit before you design the autonomy. Human handoff is not a failure path bolted on at the end; it is a named part of the system with its own triggers: low retrieval confidence, a tool error, an action outside the preapproved set, a value above a threshold. Write down what the person sees, when they see it, and what they can stop. Oversight requirements for higher-risk systems assume exactly this, and it is far cheaper to build in than to retrofit.

The test to run before you pick a shape

Take your twenty most recent real cases and write the steps each one would need. Count the distinct sequences. One or two means retrieval or a single call is probably the answer. A handful means a workflow with branches, and the leftovers should route to a person rather than justify autonomy. If the sequences barely repeat, ask the two follow-up questions before building an agent: can you check the result cheaply, and can you bound what a wrong action costs? If either answer is no, the honest next move is to narrow the problem until both are yes.