The team integrated Claude Mythos 5 on launch day. That is the kind of thing a well-run engineering team does: a new frontier model drops, you check the benchmarks, test it against your production workload, and start integrating. By day three they had tuned their prompts, updated their system prompt templates, and filed a Jira to migrate the rest of the pipeline.
By day four, the model was gone.
The US government had deemed it a national security risk. The Commerce Department barred foreign access. Anthropic, unable to verify user nationality in real time, took the model offline for everyone. Three days of integration work, gone. Not because of a bug in their code or a vendor contract dispute, but because of a regulatory decision made by people who have never heard of this startup and will never hear of it.
This Is a New Kind of Vendor Risk
Traditional vendor lock-in is a contract problem. You have built your product on top of AWS, or Stripe, or Salesforce, and switching costs are high enough that you stay. Those services do not disappear overnight. Their SLAs have teeth. Their APIs have deprecation timelines measured in years, announced months in advance.
AI model lock-in is different. It is behavioral, not contractual. The risk is not just that the API might go away. It is that the specific model you have calibrated against, the one whose tone your users are used to, the one your prompt templates assume, can be gone in 72 hours with no recourse. Your SLA with the provider does not cover a government export control. Your committed spend does not cover a national security determination. Your roadmap did not account for the possibility that the model you shipped against simply would not exist next week.
The Mythos 5 incident was an extreme case, but it crystallized something engineering teams have been quietly managing for two years: when you build production features on someone else’s model, you hand over a piece of your stack’s reliability to a party who does not share your incentives and cannot be bound by your contracts.
The Three Flavors of Model Risk
When teams talk about vendor risk for AI APIs, they focus on outages. It is the most visible failure mode. On October 6 this year, OpenAI had a 36-minute degraded performance window. In June, Anthropic had a full outage that affected Claude Chat for several hours. These feel dramatic when they happen and they are worth having fallbacks for.
But outages are the easiest risk to plan for. You add a circuit breaker. You set a timeout and return a graceful error message. You route to a backup model for simple tasks during the window. The operational pattern is familiar because you have done it for databases and third-party payment APIs. The pattern does not change because the dependency happens to be an LLM.
The harder risks are the quieter ones.
Silent behavior drift is the one that shows up in user feedback before your metrics catch it. The model you shipped against in January handles ambiguous inputs one way. By March, without a changelog entry, it handles them differently. Your evals do not catch it because your evals were written against the old behavior. Your customers notice something changed but cannot articulate it. The product feels slightly off. You spend two sprint cycles investigating before someone runs a systematic comparison and finds the divergence in the model’s response patterns.
A 2 percent shift in how a model formats a structured response is invisible in testing. At 10,000 daily requests, it surfaces in your downstream parsers as a category of silent failure that took weeks to root-cause.
Deprecation looks manageable until it is not. Providers announce deprecation with weeks or months of notice, which sounds reasonable. But notice is not the same as capacity. Migrating a production LLM application to a new model means re-running your full eval suite, adjusting prompts calibrated to the old model’s specific response patterns, potentially updating downstream parsers if output structure shifted, and re-validating with users. That work does not happen in a sprint. When the migration lands in the same quarter as a product deadline, something gets cut, and usually it is the validation.
And then there is the category the Mythos 5 incident represents: abrupt shutdown for reasons completely outside the normal vendor relationship. Regulatory action, national security determinations, export controls. A party with no contractual relationship with you makes a decision that immediately affects your production stack. Your legal agreements with the provider do not reach this.
What “Building for Resilience” Actually Means
The conventional response to this problem is “use an abstraction layer.” Build your application so the model is a configuration parameter, not a hard dependency, and you can swap it out without touching business logic. That is the right direction and it is not enough on its own.
The issue is not just that your model is hardcoded. The issue is that your prompts, your evals, your output parsers, and your guardrails are all calibrated to a specific model’s behavior. A 1 percent change in how the model formats structured output can break a downstream parser you did not know was load-bearing. The real switching cost is not the three-line config change. It is the three weeks of re-calibration afterward.
The teams who handle this well have a different mental model for what they are building. They do not think “we are building an application that uses Claude.” They think “we are building an application that the model tier plugs into.” The business logic, the evals, the routing rules, the context management, the tool definitions: all of that lives in infrastructure they own and version-control. The model is the last piece plugged in.
A useful framing here comes from an analysis of production AI agent codebases: only 1.6 percent of the code is actual AI decision logic. The other 98.4 percent is the infrastructure around it, permissions, context management, tool routing, error handling. If your team thinks of itself as “building with Claude,” that 1.6 percent is running the show. If your team thinks of itself as “building AI infrastructure that can work with any model,” the 98.4 percent is the moat.
The Practical Version
Model-agnostic architecture sounds like an enterprise concern. In practice, three habits make a significant difference for a startup team and none of them require rebuilding your stack.
Keep evals model-specific but run them on model changes, not only on feature changes. When your provider releases an update or you migrate to a new model, your eval suite runs before anything touches production. Most teams run evals when they change prompts. Fewer run them when the model changes underneath their prompts, which is the gap that silent drift exploits.
Write prompts that do not depend on output formatting quirks. If your parser assumes the model will always return JSON in a specific structure, document that assumption explicitly and build a fallback for when it does not hold. Brittle parsers are the most common silent failure point after a model update, and they are almost always invisible until something downstream breaks.
For your most critical flows, know your fallback before you need it. Pick a second model, test it against your production eval suite, and keep that eval result current. You do not have to dual-run in production. You need to know, when something goes wrong, whether your fallback is deployable in hours or in weeks. Building resilient AI-powered features requires knowing that answer before the incident, not during it.
None of this requires switching away from a single primary model or over-engineering your stack. It requires deciding, early in the build, that the model is not the foundation of your application. It is a capable, somewhat unpredictable component that you have chosen to use today and that you can replace if you need to.
The Honest Version of This Problem
There is a version of this advice that collapses into “just build everything on open-source models so you control the whole stack.” That is one valid path, and for certain industries and compliance requirements, it is the right one.
Most startup teams are not in that position and pretending they are does them no favors. Frontier models are genuinely better at a lot of tasks. Running your own models adds infrastructure cost and operational complexity that most seed-to-Series-B teams cannot absorb alongside their core product work. The goal is not to avoid hosted model APIs. It is to use them without giving away control of the layer above them.
The teams that got hit hardest by the Mythos 5 shutdown had one thing in common: they had been building fast, which is the right instinct, but building fast against a model rather than around one. The integration work was load-bearing. The prompts were in a single file. The evals existed but they were mostly “does it produce output” rather than “does it produce the right output regardless of model version.” When the model vanished, there was nowhere to redirect.
In AI consulting engagements, this tends to surface not as a vendor risk conversation but as an architecture conversation. You are not trying to hedge against your model provider. You are trying to build software that behaves predictably in an environment where one of its dependencies changes faster and less predictably than any dependency you have managed before. That reframe matters. It shifts the response from “protect yourself from the vendor” to “build software that is honest about its dependencies.”
What Is Actually Possible
A production AI stack built with this approach does not have to be complex. It has to be honest about where the boundaries are. Your application logic is yours. Your evals are yours. Your context and memory management are yours. The model is something you have plugged in, tested, and confirmed you can swap if you have to.
That is a realistic, achievable state for a startup engineering team. It does not require a multi-year platform project. It requires deciding early that the model is not the foundation of your application. Early digital transformation work that gets this distinction right avoids an entire category of incident that most teams only encounter after they have already been burned.
The model was gone in three days. The teams that were back up in hours had made a different architectural decision. Not a smarter one, just an earlier one