Customer support · Production

Doctolib improves customer care case deflection by 20% with a RAG system built on GPT-4o and OpenSearch

The problem

Doctolib's customer care relied on rules-based bots that could not adapt to users or their context. Standard LLMs lacked access to private and recent data, limiting accuracy. A RAG system was pursued to address this gap but initially suffered from insufficient results and latency exceeding one minute.

First attempt

The initial vanilla RAG implementation did not produce good enough results on its own, and response latency reached up to one minute, making the system impractical for real users.

Workflow diagram · grounded in source
1
User query submitted
trigger
“The basic RAG pipeline involves embedding a user query”
2
ML classifier gates request
ai_action
“we developed a Machine Learning classifier model to predict if our system could answer the user query. It diminished the reach of our tool but made it more precise and impactful”
3
FAQ document retrieval
ai_action
“OpenSearch as the Vector Database containing the embeddings of the chunks of the FAQ articles”
4
Re-ranking applied
ai_action
“we improved it by implementing a re-ranking brick”
5
LLM answer generation
ai_action
“passing the documents to an LLM for generation of an answer grounded in the retrieved context. We currently use GPT-4o as a LLM”
6
Customer care case deflected
output
“customer care cases deflection with a 20% reduction. Customer care agents can now focus on more complex cases”
Reported outcome

The improved RAG system achieved a 20% reduction in customer care cases reaching agents and reduced response latency from one minute to less than five seconds, enabling customer care agents to focus on more complex cases.

Reported metrics
Customer care cases deflection20%
Response latencyfrom a latency of 1 min to less than 5s
Reported stack
GPT-4oAzure OpenAI serviceOpenSearchRagas
Source
https://medium.com/doctolib/part-1-from-retrieval-augmented-generation-rag-to-agents-doctolibs-journey-to-revolutionize-d34610eeb550
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The improved RAG system achieved a 20% reduction in customer care cases reaching agents and reduced response latency from one minute to less than five seconds, enabling customer care agents to focus on more complex ca…

What tools did this team use?

GPT-4o, Azure OpenAI service, OpenSearch, Ragas.

What results were reported?

Customer care cases deflection: 20%; Response latency: from a latency of 1 min to less than 5s (source-reported, not independently verified).

What failed first in this deployment?

The initial vanilla RAG implementation did not produce good enough results on its own, and response latency reached up to one minute, making the system impractical for real users.

How is this customer support AI workflow structured?

User query submitted → ML classifier gates request → FAQ document retrieval → Re-ranking applied → LLM answer generation → Customer care case deflected.