Ecommerce ops · Production

Mercado Libre builds multi-LLM orchestration pipeline for product matching at 95% precision and sub-$0.001 cost per request

The problem

Mercado Libre's product-matching pipeline required manual human oversight for high-stakes use cases such as pricing because the existing ML model was not accurate enough to operate fully autonomously.

First attempt

An initial three-step LLM pipeline produced only 38% precision and 13% recall, far below the existing ML model's performance. Even the sophisticated ML model combining advanced NLP, multiple embeddings, and attribute enrichment could not fully automate the process.

Workflow diagram · grounded in source
1
Select target item
trigger
“Get the Target Item: Select the specific item requiring matching”
2
Encode features with embeddings
ai_action
“Use Different Open Source Embedding Models: Employ a variety of embedding models to encode the features of each item effectively”
3
Find candidates via ANN
ai_action
“Find Candidates with ANN Algorithms: Deploy Approximate Nearest Neighbors (ANN) algorithms to quickly identify potential matching candidates”
4
ML model scores candidates
ai_action
“Evaluate Candidates with an ML Model: Process these candidates through a machine learning model to calculate match probabilities”
5
Human review for sensitive cases
human_review
“Conduct Human Oversight: In certain scenarios, especially for sensitive products and use cases, a human reviews the ML model's results for confirmation”
6
Orchestrated LLM pipeline
ai_action
“Our system evolved from an initial 2-node configuration to a sophisticated 7-node architecture with enhanced capabilities. Beyond mere expansion, we implemented significant improvements, including adaptive prompts, context-aware decision…”
7
Iterative failure-case refinement
feedback_loop
“running the prompt on a sample of training examples, identifying failure cases where the results don't meet expectations, modifying the prompt to address these issues, and then generalizing it to prevent overfitting and ensure it works m…”
Reported outcome

After iterative orchestration design and prompt engineering, Mercado Libre achieved 95% precision and at least 50% recall at a cost of less than $0.001 per request, reaching human-level performance with an autonomous method scalable across millions of items.

Reported metrics
initial LLM pipeline precision38%
initial LLM pipeline recall13%
Precision after prompt refinement65%
Recall after prompt refinement79%
Show all 8 reported metrics
initial LLM pipeline precision38%
initial LLM pipeline recall13%
precision after prompt refinement65%
recall after prompt refinement79%
final target precision95%
final target recallat least 50%
cost per requestless than $0.001
system architecture node count2-node to 7-node
Reported stack
VerdiANNvector databasesembedding models
Source
https://medium.com/mercadolibre-tech/tale-of-a-prompt-development-c133081bca1e
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

After iterative orchestration design and prompt engineering, Mercado Libre achieved 95% precision and at least 50% recall at a cost of less than $0.001 per request, reaching human-level performance with an autonomous…

What tools did this team use?

Verdi, ANN, vector databases, embedding models.

What results were reported?

initial LLM pipeline precision: 38%; initial LLM pipeline recall: 13%; Precision after prompt refinement: 65%; Recall after prompt refinement: 79% (source-reported, not independently verified).

What failed first in this deployment?

An initial three-step LLM pipeline produced only 38% precision and 13% recall, far below the existing ML model's performance.

How is this ecommerce ops AI workflow structured?

Select target item → Encode features with embeddings → Find candidates via ANN → ML model scores candidates → Human review for sensitive cases → Orchestrated LLM pipeline → Iterative failure-case refinement.

Related ecommerce ops cases