Customer support · Production

Demystifying RAG: Enhancing LLMs with Real-Time Knowledge Retrieval

The problem

Large language models have static, often outdated training data, are expensive and slow to recalibrate, and are prone to producing fabricated answers, making them unreliable for domain-specific or current-knowledge queries.

First attempt

Databricks' Data Intelligence Platform lacks support for conditional filtering and boosting, causing it to underperform in relevance calculation during retrieval and making it insufficient on its own for a production-grade RAG implementation.

Workflow diagram · grounded in source
1
Rep submits query
trigger
“Help Call Center Service Representative and Field Service Representative Become More Effective at Their Job When Answering Customer Calls”
2
Retrieve relevant documents
ai_action
“done by retrieving data/documents relevant to a question or task and providing them as context to augment the prompts to an LLM to improve generation”
3
LLM generates grounded response
output
“Outputs can include citations of original sources allowing human verification”
4
Rep applies resolution
human_review
“They tried the same fix from that old case and it fixed the truck!”
Reported outcome

RAG implementations progressed from proof-of-concept to production roll-out, with a field service case illustrating how a previously stalled truck repair was resolved in 15 minutes after the RAG solution surfaced a matching historical case.

Reported metrics
Time to resolution (field service case)15 minutes
case open duration before RAG resolutiontow month
Reported stack
CoveoLucidworksGPT 3.5 turbo
Source
https://mlops.community/blog/what-on-earth-is-rag
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

RAG implementations progressed from proof-of-concept to production roll-out, with a field service case illustrating how a previously stalled truck repair was resolved in 15 minutes after the RAG solution surfaced a ma…

What tools did this team use?

Coveo, Lucidworks, GPT 3.5 turbo.

What results were reported?

Time to resolution (field service case): 15 minutes; case open duration before RAG resolution: tow month (source-reported, not independently verified).

What failed first in this deployment?

Databricks' Data Intelligence Platform lacks support for conditional filtering and boosting, causing it to underperform in relevance calculation during retrieval and making it insufficient on its own for a production-…

How is this customer support AI workflow structured?

Rep submits query → Retrieve relevant documents → LLM generates grounded response → Rep applies resolution.