Call center ai · Production

AT&T achieves 90% cost savings with fine-tuned small language models for call center operations

The problem

AT&T processes 15 million customer calls annually but needed a cost-effective way to extract actionable insights from transcribed interactions; large language models were too resource-intensive and expensive to deploy at scale.

Workflow diagram · grounded in source
1
Call transcripts collected
trigger
“The company handles 15 million customer calls annually, generating a vast amount of recorded, transcribed, and summarized interactions”
2
Distill LLMs into SLMs
ai_action
“AT&T distilled large language models, such as GPT-4, into three smaller fine-tuned open-source models”
3
Fine-tune Danube on call data
ai_action
“For 20 key categories in their 80-label classification system, they utilized H2O.ai's Danube 1.8B, a cutting-edge small language model fine-tuned with H2O LLM Studio”
4
Ensemble classifies transcripts
ai_action
“By combining 10 categories from Llama, 20 from Danube, and 50 from the classifier, they reduced costs to 35% of the previous solution”
5
Actionable insights extracted
output
“extract actionable insights from customer interactions (e.g., identifying instances where customers scheduled appointments but technicians did not show up)”
Reported outcome

Deploying fine-tuned small language models yielded 90% lower cost, 3x faster processing, 75% latency improvement, and 5x scalability, while the ensemble achieved 91% accuracy closely matching the previous solution.

Reported metrics
Cost reduction90%
Processing speed improvement3x
Latency improvement75%
Scalability increase5x
Show all 8 reported metrics
cost reduction90%
processing speed improvement3x
latency improvement75%
scalability increase5x
model accuracy91%
cost relative to previous solution35% of the previous solution
Danube cost share10%
customer calls per year15 million
Reported stack
H2O.aiH2O DanubeH2O LLM StudioGPT-4Llama
Source
https://h2o.ai/case-studies/att-call-center/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Deploying fine-tuned small language models yielded 90% lower cost, 3x faster processing, 75% latency improvement, and 5x scalability, while the ensemble achieved 91% accuracy closely matching the previous solution.

What tools did this team use?

H2O.ai, H2O Danube, H2O LLM Studio, GPT-4, Llama.

What results were reported?

Cost reduction: 90%; Processing speed improvement: 3x; Latency improvement: 75%; Scalability increase: 5x (source-reported, not independently verified).

How is this call center ai AI workflow structured?

Call transcripts collected → Distill LLMs into SLMs → Fine-tune Danube on call data → Ensemble classifies transcripts → Actionable insights extracted.

Related call center ai cases