Back office ops · Production

Once Upon a Chat Bot: The Ada Story at leboncoin

The problem

leboncoin wanted to leverage LLMs internally but could not use public platforms like ChatGPT due to data security and privacy risks, particularly the risk of confidential or client data being exposed through external services.

First attempt

The initial attempt to self-host Meta's Llama 2 on AWS was complex, slow for chat, and more expensive than a managed alternative. A later pilot of Onyx as a replacement was rejected due to infrastructure instability from its Vespa database and limited customization.

Workflow diagram · grounded in source
1
Employee submits question
Trigger
Employees submit questions to Ada through dedicated Slack apps or a standalone web interface.
▾ source quote
“we brought Ada directly into Slack, by integrating the multiple assistants into specific Slack Apps”
2
Route to domain assistant
Routing
The query is directed to a dedicated domain-specific assistant built for the relevant internal data source.
▾ source quote
“we created a dedicated assistant for each internal data source — each with its own design and purpose”
3
Query rephrasing
Ai action
The user's question is run through a lightweight keyword rephraser to improve compatibility with the retrieval engine.
▾ source quote
“before hitting OpenSearch, we run the user's question through a lightweight English keyword rephraser, making the query more compatible with OpenSearch's index”
4
Document retrieval
Ai action
Relevant document chunks are retrieved via semantic search in a Postgres vector database or keyword search via OpenSearch.
▾ source quote
“Documents were chunked, embedded and stored in a Postgres vectorDB, enabling semantic retrieval based on similarity”
5
Reranking with Cohere
Ai action
Cohere's Rerank 3.5 model reorders retrieved chunks by evaluating each directly against the user's query.
▾ source quote
“we chose Cohere's Rerank 3.5 model, which leverages a technique called cross-encoding. In this approach, the model evaluates each document (or chunk) directly in relation to the user's query”
6
Answer generation by Claude
Ai action
Anthropic Claude models via AWS Bedrock generate an answer grounded in the retrieved context.
▾ source quote
“Our assistants rely on Anthropic Claude models via AWS Bedrock”
7
Answer delivered to employee
Output
Employees receive results as ephemeral Slack messages.
▾ source quote
“receive the results as ephemeral messages”
8
Evaluation and iteration
Feedback loop
The team evaluates often and iterates quickly using Langsmith.
▾ source quote
“We evaluated often and iterated quickly thanks to Langsmith. Its asynchronous evaluation was a real game changer (reducing the time for evaluating a dataset of 120 examples from 30min to 3min)”
Reported outcome

Ada became a trusted internal assistant before being sunset in Q1 2025.
The project delivered significant GenAI expertise to leboncoin's ML team. Transition is underway to ChatGPT Enterprise, with key features being ported via MCP connectors and Custom GPTs.

Reported metrics
Backstage retrieval context relevance0.63 to 0.73
Correct source link returned70%
Evaluation dataset time reduction30min to 3min
Correctness improvement from rule-based rephraser10%
Show all 7 reported metrics
Backstage retrieval context relevance0.63 to 0.73
correct source link returned70%
evaluation dataset time reduction30min to 3min
correctness improvement from rule-based rephraser10%
correctness improvement from LLM-based rephraser3–4%
reranker impact on system latencyminimal
reranker impact on answer quality and reliabilitysubstantial
Reported stack
ClaudeClaude 2Claude SonnetAWS BedrockLlama 2PostgresOpenSearchLangsmithAirflowOnyxn8nConfluenceLumappsBackstageRAGChatGPT EnterpriseMCPVespaSlack
◆ Does this fit your context?

Compare to your context

Tell us your scale, team, and constraints. We'll show what changes at your size, what fails at your scale, and whether this case is a fit, needs adaptation, or won't scale to you. Free demo, no signup.

Compare to your context →
~30 seconds · free
Source
https://medium.com/leboncoin-tech-blog/once-upon-a-chat-bot-the-ada-story-at-leboncoin-1a4c52000d82
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Ada became a trusted internal assistant before being sunset in Q1 2025.

What tools did this team use?

Claude, Claude 2, Claude Sonnet, AWS Bedrock, Llama 2, Postgres, OpenSearch, Langsmith, Airflow, Onyx.

What results were reported?

Backstage retrieval context relevance: 0.63 to 0.73; Correct source link returned: 70%; Evaluation dataset time reduction: 30min to 3min; Correctness improvement from rule-based rephraser: 10% (source-reported, not independently verified).

What failed first in this deployment?

The initial attempt to self-host Meta's Llama 2 on AWS was complex, slow for chat, and more expensive than a managed alternative.

How is this back office ops AI workflow structured?

Employee submits question → Route to domain assistant → Query rephrasing → Document retrieval → Reranking with Cohere → Answer generation by Claude → Answer delivered to employee → Evaluation and iteration.

Related back office ops cases

WHAT TO DO WITH THIS

Now compare it to your context

This case is one data point. Whether its pattern fits you depends on your volumes, your stack, and your exception load — that comparison is the step no case study can do for you.

Browse the Back office ops cases →