Self-host vs API: the cost crossover most teams get wrong

I watched two teams make opposite versions of the same mistake within a few months of each other, which is the best way to understand a decision.

The first was a platform team at a startup with a light AI workload. They self-hosted an open model from the start because the story that self-hosting is cheaper is everywhere and it sounds obviously true. At their volume it was not. The per-request cost of a hosted API would have been small, because they made few requests. Instead they were paying for GPU instances, and more quietly, they were paying engineers to keep those instances patched, monitored, and alive. At low volume the infrastructure and the operational overhead dwarfed what the requests themselves would have cost on an API. They were losing money to save money.

The second was a platform team at a larger company running heavy, steady, high-volume traffic through a hosted API. They had never revisited that choice either. At their volume the per-request price added up to far more than running their own inference would have cost, even after accounting for the ops burden. They were overpaying by staying on the convenient path long after it stopped being the cheap one.

Same error, mirrored. Both teams picked a side based on a general belief and never checked it against their actual volume.

There is a crossover, and it moves with your real volume

Self-hosting and calling an API have different cost shapes. An API is mostly variable cost: you pay per request, near zero when idle, and it climbs steadily as volume grows. Self-hosting is mostly fixed cost: you pay for capacity and for the people who operate it whether or not you use it, and that base does not shrink when traffic is light. Plot those two shapes and they cross. Below the crossover, the API is cheaper because you are not carrying the fixed base. Above it, self-hosting is cheaper because the fixed base is spread across enough requests to beat the per-request price.

The startup was well below the crossover and self-hosted anyway. The larger company was well above it and stayed on the API anyway. Neither had found the crossover, because finding it takes real numbers: your actual request volume, the real per-request API price at your token sizes, the true cost of the GPU capacity you would need, and honestly, the engineering time to run it. That last one is the piece teams leave out, and it is often what moves the crossover. Self-hosting is never free to operate.

We did the math with each of them. The startup moved to a hosted API and their AI costs dropped, and a couple of engineers got their week back. The larger company built out self-hosted inference for their steady base load, kept the API for spillover and spikes, and cut their per-request cost sharply. The right answer was different for each, and both answers came from the same calculation.

What good teams do

Find the crossover with real numbers: your volume, real API pricing at your token sizes, true capacity cost, and the engineering time to operate self-hosted infrastructure.
Count the ops burden honestly. Patching, monitoring, on-call, and idle capacity are real costs of self-hosting.
Match the choice to where your volume actually sits, not to a general belief about which is cheaper.
Consider a hybrid: self-host the steady base load, use the API for spikes and spillover, so you are not sized for peak.
Revisit the decision as volume grows. The right answer at launch is often the wrong one a year later, in either direction.

Neither team was foolish. They had each made a reasonable-sounding default and then never questioned it as their volume moved. The decision was frozen while the thing it depended on kept changing.

How we approach it at Density Labs

The self-host versus API question gets answered once, by reflex, and then quietly costs money in whichever direction the reflex went. Our AI Opportunity Assessment is a fixed two-week engagement, priced at $2,500, where we scope the real production work before you build. We run the crossover math on your actual volume and token sizes, count the operational cost honestly, and tell you which side of the line you are on, including whether a hybrid is the real answer. You leave with a number, not a belief.

Neither self-hosting nor the API is cheaper in general. There is a crossover, it depends on your real volume, and the only way to know which side you are on is to do the math before you commit.