Ecommerce ops · Production

Delivery Hero improves multilingual search with few-shot LLM translations across 68 countries

The problem

Delivery Hero's global search solution spanning 68 countries faced challenges with dialectal variations, transliterations, and spelling errors that traditional machine translation tools could not accurately handle, particularly failing to capture user intent and regional language nuances.

First attempt

Commercial machine translation tools Google Translate and DeepL generally performed under 80% accuracy on Arabic grocery search queries, failing to capture user intent and context as effectively as LLMs.

Workflow diagram · grounded in source
1
Multilingual query received
trigger
“The search solution is served from Berlin to all 68 countries, overcoming the challenges of multilingual data in English, Spanish, Arabic, Turkish, Greek, Chinese, Thai, and many more”
2
Few-shot LLM translation
ai_action
“The translation task in our project involves multiple strategies, such as transliteration, direct translation, and translations based on regional dialects. LLMs must also handle spelling variations and meaning differences based on a give…”
3
Majority voting validation
validation
“we implemented a majority voting system to increase the confidence of our translations. Given that LLMs can sometimes produce creative but unreliable responses, we ran each query through the model three times and selected the most freque…”
4
Translation-enhanced retrieval
integration
“In our ES implementation, we enhance retrieval by considering the original query and its translations (if available). This multi-faceted approach helps improve recall by capturing a wider range of relevant results”
5
Hybrid search results returned
output
“search results are retrieved from both ES and a Vector Search database, and the results are mixed”
Reported outcome

LLM-based few-shot translation using Gemini achieved over 90% accuracy for restaurant-related translations, and A/B testing showed positive improvements in user engagement; the translations are now in production for Talabat and Hungerstation.

Reported metrics
ChatGPT-Turbo translation accuracy86%
Gemini translation accuracyabout 82%
Commercial translation tools accuracyunder 80%
Restaurant-related translation accuracy with LLMover 90%
Show all 5 reported metrics
ChatGPT-Turbo translation accuracy86%
Gemini translation accuracyabout 82%
Commercial translation tools accuracyunder 80%
Restaurant-related translation accuracy with LLMover 90%
User engagement improvementpositive improvements in user engagement
Reported stack
GeminiChatGPT-TurboGoogle TranslateDeepLElastic SearchChatGPT
Source
https://tech.deliveryhero.com/blog/how-we-improved-multilingual-search-with-few-shot-llm-translations/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

LLM-based few-shot translation using Gemini achieved over 90% accuracy for restaurant-related translations, and A/B testing showed positive improvements in user engagement; the translations are now in production for T…

What tools did this team use?

Gemini, ChatGPT-Turbo, Google Translate, DeepL, Elastic Search, ChatGPT.

What results were reported?

ChatGPT-Turbo translation accuracy: 86%; Gemini translation accuracy: about 82%; Commercial translation tools accuracy: under 80%; Restaurant-related translation accuracy with LLM: over 90% (source-reported, not independently verified).

What failed first in this deployment?

Commercial machine translation tools Google Translate and DeepL generally performed under 80% accuracy on Arabic grocery search queries, failing to capture user intent and context as effectively as LLMs.

How is this ecommerce ops AI workflow structured?

Multilingual query received → Few-shot LLM translation → Majority voting validation → Translation-enhanced retrieval → Hybrid search results returned.

Related ecommerce ops cases