Back office ops · Production

Once Upon a Chat Bot: The Ada Story at leboncoin

The problem

leboncoin wanted to leverage LLMs internally but could not use public platforms like ChatGPT due to data security and privacy risks, particularly the risk of confidential or client data being exposed through external services.

First attempt

The initial attempt to self-host Meta's Llama 2 on AWS was complex, slow for chat, and more expensive than a managed alternative. A later pilot of Onyx as a replacement was rejected due to infrastructure instability from its Vespa database and limited customization.

Workflow diagram · grounded in source
1
Employee submits question
trigger
“we brought Ada directly into Slack, by integrating the multiple assistants into specific Slack Apps”
2
Route to domain assistant
routing
“we created a dedicated assistant for each internal data source — each with its own design and purpose”
3
Query rephrasing
ai_action
“before hitting OpenSearch, we run the user's question through a lightweight English keyword rephraser, making the query more compatible with OpenSearch's index”
4
Document retrieval
ai_action
“Documents were chunked, embedded and stored in a Postgres vectorDB, enabling semantic retrieval based on similarity”
5
Reranking with Cohere
ai_action
“we chose Cohere's Rerank 3.5 model, which leverages a technique called cross-encoding. In this approach, the model evaluates each document (or chunk) directly in relation to the user's query”
6
Answer generation by Claude
ai_action
“Our assistants rely on Anthropic Claude models via AWS Bedrock”
7
Answer delivered to employee
output
“receive the results as ephemeral messages”
8
Evaluation and iteration
feedback_loop
“We evaluated often and iterated quickly thanks to Langsmith. Its asynchronous evaluation was a real game changer (reducing the time for evaluating a dataset of 120 examples from 30min to 3min)”
Reported outcome

Ada became a trusted internal assistant before being sunset in Q1 2025.
The project delivered significant GenAI expertise to leboncoin's ML team. Transition is underway to ChatGPT Enterprise, with key features being ported via MCP connectors and Custom GPTs.

Reported metrics
Backstage retrieval context relevance0.63 to 0.73
Correct source link returned70%
Evaluation dataset time reduction30min to 3min
Correctness improvement from rule-based rephraser10%
Show all 7 reported metrics
Backstage retrieval context relevance0.63 to 0.73
correct source link returned70%
evaluation dataset time reduction30min to 3min
correctness improvement from rule-based rephraser10%
correctness improvement from LLM-based rephraser3–4%
reranker impact on system latencyminimal
reranker impact on answer quality and reliabilitysubstantial
Reported stack
ClaudeClaude 2Claude SonnetAWS BedrockLlama 2PostgresOpenSearchLangsmithAirflowOnyxn8nConfluenceLumappsBackstageRAGChatGPT EnterpriseMCPVespaSlack
Source
https://medium.com/leboncoin-tech-blog/once-upon-a-chat-bot-the-ada-story-at-leboncoin-1a4c52000d82
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Ada became a trusted internal assistant before being sunset in Q1 2025.

What tools did this team use?

Claude, Claude 2, Claude Sonnet, AWS Bedrock, Llama 2, Postgres, OpenSearch, Langsmith, Airflow, Onyx.

What results were reported?

Backstage retrieval context relevance: 0.63 to 0.73; Correct source link returned: 70%; Evaluation dataset time reduction: 30min to 3min; Correctness improvement from rule-based rephraser: 10% (source-reported, not independently verified).

What failed first in this deployment?

The initial attempt to self-host Meta's Llama 2 on AWS was complex, slow for chat, and more expensive than a managed alternative.

How is this back office ops AI workflow structured?

Employee submits question → Route to domain assistant → Query rephrasing → Document retrieval → Reranking with Cohere → Answer generation by Claude → Answer delivered to employee → Evaluation and iteration.

Related back office ops cases