Ecommerce ops · Production

DoorDash improves search retrieval using LLMs and RAG, increasing dish carousel trigger rate by ~30%

The problem

DoorDash's search system had to handle complex multi-requirement queries while enforcing hard retrieval rules such as dietary restrictions, but traditional query segmentation methods (PMI, n-gram analysis) fell short on complex or ambiguous queries, and embedding-based retrieval systems returned imprecise candidate sets that ignored hard constraints.

First attempt

Traditional statistical query segmentation methods (PMI and n-gram analysis) could not capture contextual relationships in complex queries, and standalone embedding-based retrieval systems returned imprecise results that ignored hard constraints such as dietary preferences.

Workflow diagram · grounded in source
1
User submits search query
trigger
“users commonly conduct searches using precise queries that compound multiple requirements”
2
LLM query segmentation
ai_action
“we prompt it to provide a structured output mapping each meaningful word segment to one of our taxonomy categories”
3
ANN candidate pre-filtering
ai_action
“using an ANN retrieval system, we retrieve the closest 100 taxonomy concepts, or candidate labels, for each search query”
4
LLM entity linking
ai_action
“We then prompt the LLM to link queries to corresponding entities from specific taxonomies such as dish types, dietary preferences, cuisines, etc.”
5
Hallucination post-processing
validation
“we developed post-processing steps to prevent potential hallucinations in the final output and ensure the validity of both our segmented queries and their linked entities”
6
Annotator manual audit
human_review
“Annotators review a statistically significant sample of the output to verify that query segments are correctly identified and accurately linked to the appropriate entities in the knowledge graph. This manual evaluation helps us detect an…”
7
Ranker signal integration
integration
“After introducing the new query understanding signals, we needed to make them available to the rankers”
8
Dish carousel results displayed
output
“DoorDash's popular dish carousel, shown in Figure 3, relies on this retrieval pipeline to display relevant results for queries that reflect a specific dish intent”
9
Ranker retraining feedback loop
feedback_loop
“with new and more diverse engagement coming in from the improved retrieval systems, we could retrain our ranker with a more comprehensive dataset”
Reported outcome

After deploying LLM-based query understanding with RAG entity linking, DoorDash achieved nearly a 30% increase in dish carousel trigger rate, more than a 2% increase in whole page relevance for dish-intent queries, and a further 1.6% WPR improvement from a retrained ranker, with same-day conversions also rising.

Reported metrics
LLM hallucination rate on segmentationless than one percent
Dish carousel trigger rate increasenearly a 30% increase over our baseline
whole page relevance (WPR) for dish-intent queriesmore than two percent increase
WPR improvement from retrained ranker1.6%
Show all 6 reported metrics
LLM hallucination rate on segmentationless than one percent
dish carousel trigger rate increasenearly a 30% increase over our baseline
whole page relevance (WPR) for dish-intent queriesmore than two percent increase
WPR improvement from retrained ranker1.6%
items retrievedsignificantly more items
same-day conversionsrise in same-day conversions
Reported stack
LLMsRAGANNBM25knowledge graph
Source
https://careersatdoordash.com/blog/how-doordash-leverages-llms-for-better-search-retrieval/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

After deploying LLM-based query understanding with RAG entity linking, DoorDash achieved nearly a 30% increase in dish carousel trigger rate, more than a 2% increase in whole page relevance for dish-intent queries, an…

What tools did this team use?

LLMs, RAG, ANN, BM25, knowledge graph.

What results were reported?

LLM hallucination rate on segmentation: less than one percent; Dish carousel trigger rate increase: nearly a 30% increase over our baseline; whole page relevance (WPR) for dish-intent queries: more than two percent increase; WPR improvement from retrained ranker: 1.6% (source-reported, not independently verified).

What failed first in this deployment?

Traditional statistical query segmentation methods (PMI and n-gram analysis) could not capture contextual relationships in complex queries, and standalone embedding-based retrieval systems returned imprecise results t…

How is this ecommerce ops AI workflow structured?

User submits search query → LLM query segmentation → ANN candidate pre-filtering → LLM entity linking → Hallucination post-processing → Annotator manual audit → Ranker signal integration → Dish carousel results displayed → Ranker retraining feedback loop.

Related ecommerce ops cases