Customer support · Production

Building voice AI at scale: Assembled's multi-agent architecture for millions of production calls

The problem

Running voice AI reliably in production is far harder than building demos: achieving high conversational quality, low latency, and near-100% uptime simultaneously — while working with rapidly evolving technology — creates significant engineering challenges.

First attempt

A single-prompt approach to voice agents — putting all context into one LLM prompt — failed to scale to enterprise production volume, and real-world deployment exposed problems with poor audio and frustrated callers that could not be anticipated from demos.

Workflow diagram · grounded in source
1
Customer initiates call
trigger
“voice agents that handle millions of real customer calls”
2
Orchestration layer determines intent
routing
“We have an entire orchestration layer around determining what the customer is asking about, how to transition from one agent to another, and how to transition back if needed.”
3
Specialized agent handles task
ai_action
“Depending on the context of the call — whether the caller is asking about a certain procedure, requesting a transfer, or asking a question relevant to the customer's knowledge base — we can transfer them to specific agents specialized fo…”
4
Parallel acknowledgment generation
ai_action
“It involves a lot of parallelization within the LLM's brain in creative ways so you can still get a really high-quality answer while making the conversation feel natural.”
5
Account or system lookup
integration
“like looking up a customer's account”
6
Live agent fallback
human_review
“ensuring that if it is impacted, calls always get to a live agent so customers still get the help they need”
7
Deep call quality auditing
feedback_loop
“We were only able to surface certain problems because we scaled quickly and did deep quality audits of our calls. That helped us see where things were working well, where there was room for improvement, and what the real limits of our sy…”
Reported outcome

Assembled's voice agents handle millions of calls per year in production using a multi-agent architecture, with deep quality auditing enabling continuous improvement in conversation quality and reliability.

Reported metrics
Production call volumemillions of calls per year
Reported stack
GPT-4o miniGemini 2.5 Flashspeech to texttext to speech
Source
https://www.assembled.com/blog/building-voice-ai-at-scale
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Assembled's voice agents handle millions of calls per year in production using a multi-agent architecture, with deep quality auditing enabling continuous improvement in conversation quality and reliability.

What tools did this team use?

GPT-4o mini, Gemini 2.5 Flash, speech to text, text to speech.

What results were reported?

Production call volume: millions of calls per year (source-reported, not independently verified).

What failed first in this deployment?

A single-prompt approach to voice agents — putting all context into one LLM prompt — failed to scale to enterprise production volume, and real-world deployment exposed problems with poor audio and frustrated callers t…

How is this customer support AI workflow structured?

Customer initiates call → Orchestration layer determines intent → Specialized agent handles task → Parallel acknowledgment generation → Account or system lookup → Live agent fallback → Deep call quality auditing.

Related customer support cases