Key takeaways
- Models are retired on schedules you can look up, so plan for replacement rather than treating it as an incident.
- One internal interface or gateway makes model swaps, fallbacks and cost tracking a configuration change.
- Prompts, evaluation sets and source data are the assets worth keeping portable, more than the model itself.
- Lock-in hides in vector stores, agent frameworks, fine-tunes and multi-year commitments, not in the model call.
- Abstraction has real costs: a lag behind new provider features, an extra component to run, and a lowest-common-denominator interface.
Most lock-in arguments are theoretical until a date appears in your inbox. A model your workflow depends on has a retirement date, the replacement behaves slightly differently, and suddenly a quiet piece of automation needs an engineer, a test cycle and a change approval. The question is not whether that happens. It is how much of your system has to move when it does.
Why do AI models change so often?
Model releases arrive far faster than most enterprise software generations, and providers retire older versions to free capacity for newer ones. This is published policy, not speculation.
On Microsoft Foundry, generally available models get a retirement date set programmatically at launch, 18 months out. Models from Anthropic, DeepSeek, Fireworks and Mistral AI follow a 12-month lifecycle instead. Subscription owners with active deployments receive at least 60 days notice, retirement dates cannot be extended, and after retirement all inference requests return an error. Microsoft also selects the official replacement model only about 90 to 120 days before retirement, on the reasoning that naming one earlier risks pointing customers at a model that is no longer the best choice by the time they need it.
Amazon Bedrock is similar in shape. Models sit in one of three states: Active, Legacy or end-of-life. Each model card carries an "EOL no sooner than" date and a Legacy notice period of either 6 months or 45 days, with most models on 6 months. Once a model enters Legacy, new customers cannot adopt it and existing customers may lose access after 15 days of inactivity. AWS is explicit that after the end-of-life date, requests fail and migration will not happen automatically.
Two practical conclusions follow. Model replacement is a scheduled maintenance activity, like a database version upgrade. And a model swap is never only a swap: Microsoft's own migration guidance points out that parameters such as temperature and maximum tokens do not map one to one across model generations, that tool definitions and output schemas often need tightening, and that the calling code itself can change.
What does model-agnostic actually mean?
Model-agnostic does not mean using every provider, or refusing to have a favorite. It means your application depends on an interface you control rather than on a particular vendor's SDK, so the choice of model is a decision you can revisit cheaply.
Three properties make it real:
- One internal interface. Application code calls your own function or service to get a completion, an extraction or an embedding. Nothing outside that module imports a provider library.
- Configuration, not code, selects the model. Which model serves which task is data: a config file or a routing rule, changeable without a release.
- Quality is measured the same way whatever the model. One evaluation set, run against any candidate, so a swap is a decision backed by evidence.
Without the third property the first two are decoration. Being able to switch models quickly is only useful if you can tell whether the new one is worse.
How does a gateway or abstraction layer work?
The pattern is ordinary. An abstraction layer translates a single request format into whatever each provider expects and normalizes the response. You can implement it as a library inside your application or as a gateway service that sits in front of the providers.
The open-source library LiteLLM is the common reference point. It offers one interface to more than 100 large language models using the OpenAI request format, and its proxy adds the controls most companies actually want: virtual keys with per-key, per-team and per-user budgets, retry and fallback logic across multiple deployments, routing and load balancing, centralized logging and cost tracking. The AI SDK takes the library route, abstracting provider differences behind a single language model specification across more than 40 providers, with switching handled by changing which provider instance you instantiate.
A gateway earns its place once more than one application uses models, because it becomes the single place to enforce budgets, capture usage per team and fail over when a provider is degraded. Microsoft's migration guidance notes the same thing from the other direction: weighted routing between two deployments is implemented in your own gateway or application layer, not by the platform.
Illustrative example: Consider a distributor running three AI workflows: order document extraction, a customer service reply drafter and an internal policy search. Each was built by a different contractor, and each imports a different provider SDK. When one provider changes an API version, only one workflow breaks, but nobody can say which models are in use or what any of them cost. Consolidating the three onto one internal interface takes a few days of work. The lasting gain is not the swap ability, it is that model usage, spend and failures finally appear in one place with one owner.
What has to stay portable besides the model call?
The model is the most replaceable part of an AI system. The assets worth protecting are the ones you built.
- Prompts and instructions. Keep them in version control, in your repository, as text you own. They encode business rules that took real effort to get right.
- Evaluation sets. The set of real cases and expected answers is what lets you compare a new model with the current one. Microsoft's migration process is built around freezing that dataset and scoring the current model first to establish a baseline. Without it you cannot migrate deliberately.
- Source data and the index. Retrieval systems turn your documents into vectors for search. What matters is that you keep the source documents and the pipeline that builds the index, so it can be rebuilt elsewhere. An open option such as pgvector, a Postgres extension that stores vectors alongside the rest of your data and supports HNSW and IVFFlat indexes, keeps that inside a database you already run.
- Tool and integration definitions. The list of actions the AI can take, and the permission checks around them, belong in your code. The open Model Context Protocol, described by its maintainers as a standard way to connect AI applications to external systems and supported across clients including Claude, ChatGPT, Visual Studio Code and Cursor, is one way to define those connections once rather than per framework.
- Logs and traces. Records of what was asked, what was answered and what it cost are how you investigate problems and prove decisions later. Keep them in your own account, with your own retention rules.
Where does lock-in actually hide?
Almost never in the completion call. It hides in the things that are expensive to rebuild.
| Where it hides | What it looks like | How to reduce it |
|---|---|---|
| Proprietary vector stores | Embeddings and chunking tuned to one managed index, with no export of the original text | Keep source documents and the indexing pipeline; treat the index as rebuildable |
| Agent frameworks | Business logic written as framework-specific chains, agents and handlers | Keep decision rules and permission checks in ordinary application code |
| Fine-tuned models | Weights that exist only on one platform | Prefer prompting and retrieval first; treat fine-tuning as a deliberate, revisitable choice |
| Managed connectors | Integrations that exist only inside one vendor's low-code tool | Own the credentials and know how the same call would be made directly |
| Contract commitments | Multi-year spend commitments or exclusivity terms | Keep terms shorter than a model generation; avoid exclusive-use clauses |
| Knowledge | One contractor who understands the prompts and routing | Require documentation and a named internal owner from the first release |
Fine-tuning deserves particular care. Microsoft treats fine-tuned workloads as out of scope for its standard migration process: they are not auto-upgraded, they carry their own training and deployment retirement clock, and moving them to a new base model means distillation or re-training rather than editing a prompt. That may still be the right decision, but it should be made knowingly.
What does abstraction cost you?
Any honest version of this argument admits the trade-offs.
- You lag behind new provider features. Structured output modes, caching behavior and reasoning controls often appear in a provider SDK before an abstraction layer supports them.
- You get the common denominator. A shared interface exposes what most providers support. Genuinely differentiated capabilities need a deliberate escape hatch.
- It is one more component to run. A gateway needs monitoring, patching and a plan for what happens when it fails, which cannot be "all AI stops."
- Abstraction does not make models interchangeable. Swapping remains a tested release. The layer removes the code work, not the evaluation work.
- It can become its own lock-in. A heavily customized internal framework is as hard to leave as a vendor's.
The reasonable position is a thin layer with an escape hatch: route everything through one interface, allow a specific workflow to call a provider directly when a capability justifies it, and record why.
A practical checklist for staying portable
- All model calls go through one internal module or gateway, and nothing else imports a provider SDK.
- Model choice per task lives in configuration that can change without a code release.
- Prompts and instructions are in your repository, under version control.
- An evaluation set of real cases exists, is frozen during a migration, and runs before any model change ships. See how to test AI before it touches your operations.
- Provider accounts are yours, with your own billing relationship and your own API keys held in a secret store.
- Retrieval source documents and the indexing pipeline are yours, and the index can be rebuilt from them.
- Retirement dates for every model in use are recorded with a named owner who tracks them.
- A fallback path exists for when the primary model or region is unavailable, and it has passed the same evaluation set.
- Spend is attributed per application or team, with hard usage limits enforced somewhere you control.
- Contract terms have been read for exclusivity and minimum commitment, and their end dates sit in the same calendar as the model retirement dates.
Nine of those ten cost days, not months, if they are decided at the start. Retrofitting them after three workflows are live is where the expense appears. For how this fits the wider production setup, see running AI in production on AWS and Azure, and for the platform decision itself, AWS or Azure for AI workloads.
How Kastling approaches model-agnostic architecture
Model-agnostic design is part of how Kastling delivers AI & Cloud Infrastructure work: cloud architecture and deployment on AWS or Azure, AI application deployment, and the data connections behind them.
In practice that means connecting the systems a client already runs rather than replacing them, deploying into the client's own AWS account or Azure subscription where that is the right answer, and setting out account ownership, access, operating costs and responsibilities in the proposal. Where a workload runs in Kastling-hosted accounts, usage is billed on a pay-as-you-go basis and the arrangement is written down rather than assumed. Data access and ownership are agreed up front, and solutions are tested against real business scenarios before they go live, which is what makes a later model change a routine release. The team is founder-led, with AWS and Azure specialists brought in as the work requires.
Questions
Does a gateway make our AI slower?
It adds a network hop, which is usually small next to the seconds a model takes to generate a response. If latency is critical, run the gateway close to your application and measure it as part of your normal performance testing rather than assuming either way. Most teams find the routing, budget and logging controls easily worth the milliseconds.
Is model-agnostic the same as multi-cloud?
No, and conflating them creates unnecessary work. Model-agnostic means your application is not welded to one model provider. Multi-cloud means running infrastructure across more than one cloud, which is far more expensive and rarely justified for a business AI workload. You can be model-agnostic entirely within AWS or entirely within Azure.
Should we still negotiate a commitment with one provider?
Often yes, because committed spend can reduce unit costs meaningfully. The point is to know what the commitment buys and what it forecloses. Keep the term short enough to survive a model generation, avoid clauses that require exclusive use, and make sure the architecture would still let you move if the economics change.
We already built on one provider SDK. How hard is it to change?
Usually less work than people fear for the call itself, and more work than expected around it. Extracting the calls into one internal module is mechanical. The real effort is re-testing prompts and output schemas against the new model, which is why an evaluation set is the prerequisite rather than a nice-to-have.