Ecommerce ops · Production

DoorDash uses LLMs to bridge behavioral silos in multi-vertical recommendations

The problem

As DoorDash expands into more verticals, most customers have deep behavioral history in only a few categories — especially restaurants — leaving them effectively cold-start in grocery, retail, and convenience. Standard recommenders have little per-SKU signal, and popularity baselines overexpose head products while pushing aside long-tail items, weakening personalization across large, sparse catalogs.

First attempt

Before prompt refinements, the LLM assigned overly generic and incorrect category tags — a user who ordered Indian food was tagged with categories like 'Sandwiches' rather than relevant fine-grained categories like 'Specialty Breads (Naan)'.

Workflow diagram · grounded in source

User behavior data as input

trigger

“We use LLMs to translate unstructured user behavior, like restaurant orders and search queries, into a structured, four level product taxonomy”

H-RAG affinity inference

ai_action

“The model first predicts broad category affinities at higher taxonomy levels (L1, L2). These high-confidence predictions then constrain the search space at deeper levels (L3, L4). The model iteratively refines its guesses, avoiding plaus…”

Confidence threshold filtering

validation

“Keep only categories with confidence ≥ 0.80. This acts as a builtin filter, removing low confidence or spurious associations”

LLM feature augmentation

integration

“We enrich the model input by concatenating LLM-derived user affinities with existing features”

Multi-task ranker output

output

“Our item ranker jointly optimizes for multiple objectives (e.g., click through rate, add to cart, purchase)”

Reported outcome

The LLM-powered framework achieved a 4.4% relative improvement in AUC-ROC and 4.8% improvement in MRR offline, confirmed with +4.3% AUC-ROC and +3.2% MRR gains in online production, while cutting total computation costs by ~80%.

Reported metrics

AUC-ROC improvement (overall, offline)4.4%

MRR improvement (overall, offline)4.8%

AUC-ROC lift (cold-start consumers)4.0%

MRR lift (cold-start consumers)1.1%

Show all 9 reported metrics

AUC-ROC improvement (overall, offline)4.4%

MRR improvement (overall, offline)4.8%

AUC-ROC lift (cold-start consumers)4.0%

MRR lift (cold-start consumers)1.1%

AUC-ROC lift (power consumers)5.2%

MRR lift (power consumers)2.2%

AUC-ROC improvement (online)+4.3%

MRR improvement (online)+3.2%

computation cost reduction~80%

Reported stack

LLMGPT 4oGPT 4o-miniH-RAG

Source

https://careersatdoordash.com/blog/doordash-llms-bridge-behavioral-silos-in-multi-vertical-recommendations/

Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The LLM-powered framework achieved a 4.4% relative improvement in AUC-ROC and 4.8% improvement in MRR offline, confirmed with +4.3% AUC-ROC and +3.2% MRR gains in online production, while cutting total computation cos…

What tools did this team use?

LLM, GPT 4o, GPT 4o-mini, H-RAG.

What results were reported?

AUC-ROC improvement (overall, offline): 4.4%; MRR improvement (overall, offline): 4.8%; AUC-ROC lift (cold-start consumers): 4.0%; MRR lift (cold-start consumers): 1.1% (source-reported, not independently verified).

What failed first in this deployment?

How is this ecommerce ops AI workflow structured?

User behavior data as input → H-RAG affinity inference → Confidence threshold filtering → LLM feature augmentation → Multi-task ranker output.