Don't Marry Your Product to One LLM Vendor

The most expensive mistake in an AI feature isn't picking the wrong model — it's picking a model so deeply that swapping it later means touching every file that calls it. On an applied-AI product we built, the fix was almost boring: one interface, three adapters behind it, and a config switch instead of a rewrite. That decision paid for itself twice — once when a provider's pricing shifted under us, and once when we actually measured cost against quality and found the expensive option wasn't the better one.

This is a write-up of that pattern: how we drew the boundary, why it's a textbook Strategy/Adapter split rather than a clever trick, how a fast/smart tier split controls cost before you've optimized anything else, why we treat the model's output as a typed API contract instead of a blob of text, and the actual benchmark — run on real production traffic — that told us where to spend the paid API key and where not to.

§ 01One interface, not one vendor

The feature-level code in the product — the translation endpoint, the extraction job, the OCR step — never imports an SDK. It imports one internal interface and calls two methods on it: complete() for text, completeVision() for anything with an image attached. Everything provider-specific — auth, request shape, response parsing, streaming quirks — lives behind that boundary and nowhere else.

// packages/core — the only shape feature code is allowed to depend on
interface AiProvider {
  name: "openai" | "anthropic" | "ollama";
  models: Record<"fast" | "smart", string>;
  complete(opts: CompleteOpts): Promise<string>;
  completeVision(opts: VisionOpts): Promise<string>;
}

That's the entire contract. No provider-specific fields leak through it — not a system_fingerprint, not a provider's particular tool-call schema, not a streaming callback shape unique to one SDK. If a method or field can't be expressed the same way across every provider you plan to support, it doesn't belong on the interface; it becomes an adapter-level concern instead. Keeping the boundary narrow is the whole trick — a wide interface just re-exposes the vendor lock-in one layer up, and now you've hidden it behind an abstraction instead of removing it.

§ 02Strategy and Adapter, not an if/else forest

This is two named patterns doing normal, unglamorous work. Strategy is the interface itself — swap the algorithm (here, "which model answers this call") at runtime without the caller knowing or caring. Adapter is how each provider gets forced into that shape. OpenAI and a local Ollama server both speak an OpenAI-compatible chat API, so one adapter factory serves both — only the base URL, the API key, and the model names differ:

// swap providers with one env var; feature code never changes
function getProvider(name = process.env.AI_PROVIDER ?? "openai") {
  switch (name) {
    case "anthropic": return createAnthropicProvider();
    case "ollama":    return createOllamaProvider();
    default:            return createOpenAiProvider();
  }
}

Claude's Messages API isn't OpenAI-compatible, so it gets its own adapter that translates the same CompleteOpts into Anthropic's request shape and normalizes the response back down to a plain string. The point isn't that every provider fits neatly — it's that the incompatibility is absorbed once, in one small file, instead of scattered across every call site that happens to need a completion. New provider shows up next quarter? Write one more adapter. Nothing upstream of the interface changes.

§ 03Two tiers, before you optimize anything else

Provider-agnostic buys you vendor flexibility. It doesn't automatically buy you a sane cost curve — for that we split every provider into two named tiers, fast and smart, and made the tier a first-class parameter alongside the provider:

• fast — the cheap, quick model. Default for bulk work: routine extraction, high-volume requests, anything where "good enough, consistently" beats "occasionally brilliant."
• smart — the frontier model. Reserved for the hard cases: ambiguous input, vision tasks, anything where a wrong answer is expensive enough that the token cost stops mattering.

The tier lives on the same models map on the interface — provider.models.fast, provider.models.smart — so picking a tier and picking a provider are two independent, orthogonal decisions instead of one combined choice per vendor. A feature calls translateText(request, "smart") for a passage it knows is tricky and gets the better model, still through the exact same interface, still swappable by the same env var. Most calls never need to say anything — fast is the default, and that default is where the majority of your token spend actually lives.

§ 04Treat the model as a typed API, not a text generator

An LLM call that returns a string is a liability the moment a second piece of code depends on its shape. Our rule, applied at every boundary: the model's structured output goes through a schema before anything else touches it. We use zod as the single source of truth for the shape — the same schema validates an API request body, an ingestion result, and a model response, so "what does a translation look like" is defined exactly once and imported everywhere, never re-typed by hand at each call site.

Concretely: the provider is asked for JSON (json: true on the request), the response is parsed, and the parsed object is passed through a TranslateResponse schema before the caller ever sees it — detected language, the translation itself, a gloss, register notes, and an array of extracted vocabulary terms, each field typed and validated, not guessed at with optional chaining three layers downstream. If the model hallucinates a field or omits a required one, that fails loudly at the boundary instead of quietly corrupting a database row two functions later.

This is a small habit with an outsized payoff: once the contract is a schema instead of a convention, swapping the model behind it — smart for fast, OpenAI for Anthropic, frontier for local — is safe by construction. The provider can change; the shape the rest of the app depends on cannot.

§ 05The benchmark that decided where the paid key goes

Provider abstraction stops being an architecture exercise the moment you actually run the comparison it enables. We had a batch job that translates long-form transcripts — broadcasts running to roughly 2,000 segments each, full of idiomatic, dialect-heavy phrasing that punishes literal translation. We ran the identical job three ways through the same interface: OpenAI's frontier tier, OpenAI's fast tier, and the zero-marginal-cost tier of our own abstraction.

The frontier model cost about $0.54 per broadcast. The fast tier landed around $0.03. The free, local-capable tier cost $0 — and on the metric that actually mattered, idiomatic phrasing rather than literal accuracy, it matched or beat the frontier model. Not "close enough." Better on specific idioms, worse on none we logged.

Free beat "frontier" on the one axis we were actually optimizing: does this sound like a person said it, not a dictionary.

That result set policy, not just cost: the free/local tier stays the default for the offline, non-latency-sensitive batch path, and the paid key is reserved for the one place cost genuinely buys something — the live, user-facing /translate call, where response time and reliability under load matter more than shaving fractions of a cent. Without the abstraction, that comparison would have meant three separate integrations to write and three code paths to maintain just to find out the expensive one wasn't earning its keep. With it, it was a config value.

§ 06What this buys you after the benchmark

The interface doesn't just save you a rewrite when a vendor changes pricing or deprecates a model — though it does that too, and both have already happened to teams we've talked to mid-build. It turns "which model should handle this" from an architecture question into a runtime parameter you can A/B, benchmark, and revisit as models and prices move, which in this category is monthly, not yearly. Pair that with schema-validated output and you've decoupled two things that usually ship tangled together: which vendor answers a call, and what shape the rest of your product is allowed to assume that answer has.

The pattern isn't exotic — Strategy, Adapter, and a validation boundary are three of the oldest ideas in the toolbox. What's easy to skip, especially under deadline pressure, is doing it before the third call site starts importing an SDK directly. We built this one into the language app from the first translation call, not after the second provider showed up, and the benchmark in §05 is the return on that early investment — a decision made in an afternoon of interface design that later saved real money on every batch run since.

— End of essay. Building an AI feature and want it swappable, not locked to one vendor? Start a project →