Building a provider abstraction so one vendor outage isn’t your outage

A team at a mid-market SaaS wired an AI feature straight into one provider’s SDK. The client type was passed around the codebase. The provider’s request and response shapes showed up in a dozen files. It worked, and for a long time nobody had a reason to change it. Then the provider had a bad afternoon, a regional outage that lasted a couple of hours, and the feature went dark for the same couple of hours. The team’s status page said degraded. The real cause was that their uptime was pinned to someone else’s.

When they sat down to add a fallback provider, they found the provider was not in one place they could swap. It was everywhere. The SDK’s objects had leaked into business logic, into tests, into serialization. There was no seam to cut. What should have been an afternoon became a week of untangling, all of it done under the pressure of “this could happen again tomorrow.”

A hard-wired provider makes their outage your outage

The core problem is coupling. When your code speaks the provider’s dialect directly, in many places, you have no place to stand when you want to fail over or switch models. The provider is not a detail you can change. It is woven through the product. So a vendor incident becomes your incident, and a vendor price change or deprecation becomes a migration project instead of a config edit.

A thin abstraction fixes this. One small interface that says what your product needs from a model: send this prompt, get this kind of answer back. Behind it, one adapter per provider. Your code depends on the interface. The provider lives behind it, in one file, swappable.

With that seam in place, a fallback is a router: try the primary, and on failure or timeout, try the secondary. An outage becomes a few seconds of extra latency instead of a few hours of downtime. A deprecation becomes an edit to one adapter. A price shift becomes a business decision, not an engineering emergency.

Where this goes wrong in the other direction

The equal and opposite mistake is building the grand abstraction on day one. I have watched teams design an elaborate provider framework before they had shipped a single feature, complete with plugin systems and capability negotiation for providers they might use someday. That is speculative work that pays for problems you do not have yet. It slows the first feature, and the abstraction usually turns out wrong because it was designed before anyone knew what the product actually needed.

The right size is the smallest seam that lets you swap. A narrow interface shaped by what one real feature needs, and one adapter behind it. You add the second adapter when you have a real reason: a fallback you actually want, a model that is genuinely better for a case, a price you want to escape. You widen the interface only when a concrete need pushes on it.

What teams that stay swappable do

They keep provider SDK types out of business logic, behind a narrow interface they own.
They shape that interface around what their features need, not around everything a provider could theoretically offer.
They add a router with fallback only when a second provider earns its place, and they test the failover path on purpose.
They resist building for providers they are not using yet, and let real needs drive every widening of the seam.

How we approach it at Density Labs

In an AI Opportunity Assessment, our fixed two-week engagement at $2,500, we trace how deep the provider runs into the code. Is there a seam, or has the SDK leaked everywhere? We are just as alert to the opposite: a heavy abstraction built ahead of any need, slowing the team down for flexibility it never uses.

We help find the right size for where the product actually is. Usually that is a thin interface and one adapter now, with a clear, cheap path to a second when there is a reason. We do this before more features pile onto a hard-wired provider, because each one deepens the coupling and raises the cost of the day you finally need to switch. The Diagnose scopes that real production work.

Put a thin seam between your product and the vendor. Not zero, so an outage is theirs and not yours. Not a framework, so you are not paying today for a flexibility you may never need.