Back office ops · Production

Assembled builds automated LLM fallback system achieving 99.97% effective AI uptime

The problem

LLM provider outages caused multiple customer-impacting incidents at Assembled, and manual model switchovers were slow, stressful, and unable to handle partial degradations — leaving the team reactive rather than resilient.

First attempt

A first attempt using on-call engineers with an accessible configuration switch failed because blanket switches broke nuanced per-model routing, took too long, and could not reliably classify partial versus full outages.

Workflow diagram · grounded in source
1
Provider failure detected
trigger
“This enabled instant failovers when providers become unavailable”
2
Streaming state check
validation
“we only attempt a fallback if streaming hasn't yet begun”
3
Category-consistent fallback routing
routing
“If GPT-4.1-Mini fails, we fall back to Claude 3.5 Haiku (both "Fast" category), then Gemini 2.5 Flash. This ensures users get equivalent capability levels regardless of which provider ultimately handles the request.”
4
Fallback model processes request
ai_action
“Our system detects failures and switches providers within milliseconds, eliminating the 5+ minute manual switchover delays that previously caused customer-visible outages”
5
Response delivered to user
output
“customers experienced near-zero impact with request failure rates below 0.001%”
Reported outcome

Automated fallbacks achieved 99.97% effective uptime on AI model responses, reduced average failover time from 5+ minutes to hundreds of milliseconds, and eliminated manual interventions during provider outages — with request failure rates below 0.001% during a recent multi-hour outage.

Reported metrics
effective AI uptime99.97%
Average failover time after automationhundreds of milliseconds
Average failover time before automation5+ minutes
Manual interventions during provider outageszero
Show all 9 reported metrics
effective AI uptime99.97%
average failover time after automationhundreds of milliseconds
average failover time before automation5+ minutes
manual interventions during provider outageszero
request failure rate during multi-hour provider outagebelow 0.001%
prompt development time increase due to evaluation overhead20–30%
OpenAI reported uptime99.80%
Anthropic reported uptime99.58%
potential LLM provider downtime per monthover 3 hours
Reported stack
OpenAIAnthropicGPT-4.1-MiniClaude 3.5 HaikuGemini 2.5 Flash
Source
https://www.assembled.com/blog/your-llm-provider-will-go-down-but-you-dont-have-to
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Automated fallbacks achieved 99.97% effective uptime on AI model responses, reduced average failover time from 5+ minutes to hundreds of milliseconds, and eliminated manual interventions during provider outages — with…

What tools did this team use?

OpenAI, Anthropic, GPT-4.1-Mini, Claude 3.5 Haiku, Gemini 2.5 Flash.

What results were reported?

effective AI uptime: 99.97%; Average failover time after automation: hundreds of milliseconds; Average failover time before automation: 5+ minutes; Manual interventions during provider outages: zero (source-reported, not independently verified).

What failed first in this deployment?

A first attempt using on-call engineers with an accessible configuration switch failed because blanket switches broke nuanced per-model routing, took too long, and could not reliably classify partial versus full outages.

How is this back office ops AI workflow structured?

Provider failure detected → Streaming state check → Category-consistent fallback routing → Fallback model processes request → Response delivered to user.