When just calling the API is the right architecture

There is a pull in AI projects toward architecture that looks serious. A single API call feels too plain to put in a design doc, so teams add an agent framework, a vector database, a retrieval layer, and an orchestration engine before they have shipped anything. Some features need all of that. Many need a prompt and one call, and the simple version would have shipped a month sooner and cost a fraction to run.

The pipeline that a prompt replaced

An engineering manager at a customer-support SaaS told me about a feature his team overbuilt. The job was to summarize a support ticket and suggest a category. They designed it as a multi-step agent that retrieved similar tickets, reasoned across them, and composed an answer. It worked, and it was slow, and the token bill was uncomfortable because an agentic workflow can use ten to thirty times the tokens of a single call.

During a review someone asked what would happen if they just sent the ticket text to the model with a clear instruction and no retrieval. They tried it. The plain call was nearly as good, ran in a fraction of the time, and cost a small fraction of the agent version. The similar-ticket retrieval, which had seemed obviously useful, added latency and expense without moving the quality that mattered. They shipped the one-call version and deleted the pipeline. He said the hard part was admitting the impressive design was the wrong one.

Complexity should be pulled in, not pushed

A CTO at an insurance startup gave me the discipline behind that. His rule was that every piece of architecture beyond the model call had to earn its place with evidence, not with a hunch that it would be needed. He started features at the simplest thing that could possibly work, a single call, and added a layer only when a measured problem demanded it.

Retrieval came in when the model needed facts it could not hold in context, and not before. An agent loop came in when a task genuinely required multiple dependent steps the model could not do in one pass. Each addition multiplied the cost, the latency, and the number of things that could break, so each one had to pay for itself. He had watched teams carry the weight of an agent framework for a job a prompt could do, paying that tax every request for a capability they never used.

A short test for whether the complexity is earned:

Start at one call. A single prompt to the model is the baseline. Beat it before you leave it.
Add retrieval only for facts the model lacks. If the answer is in the prompt already, a vector store is dead weight.
Add an agent loop only for real multi-step work. Dependent steps that one pass cannot do justify it. A tidy diagram does not.
Price every layer in tokens and latency. Each addition multiplies both. Make sure the quality gain is worth the multiplier.

How we approach it at Density Labs

In the AI Opportunity Assessment, our fixed two-week engagement at $2,500, we tend to push the architecture down, not up. We ask what the simplest version that could work looks like, get it running, and measure it before anyone reaches for orchestration. Often the simple version clears the bar, and the team ships something cheaper to run and easier to keep alive. When a feature truly needs the heavier machinery, that shows up as a measured gap rather than a first assumption, and the added cost is one the team chose against a number.

The plain API call is unglamorous and frequently correct. Earn your way into complexity, and make the model prove it cannot do the job alone before you build a system around it.