Ecommerce ops · Production

DoorDash builds DashCLIP: a multimodal embedding framework for CPG ad ranking and retrieval

The problem

DoorDash's ads quality team relied on categorical and numerical features that failed to capture the semantic richness of product catalogs or a deeper understanding of user interests, limiting performance on ranking and retrieval tasks.

First attempt

Off-the-shelf models lacked e-commerce domain specificity and frequently failed on short but specific queries; historical engagement data introduced position and selection bias into training labels.

Workflow diagram · grounded in source
1
Product catalog data ingestion
integration
“We curated a list of roughly 400,000 products — including their titles, images, and aisle categories — to use their catalog data for continual pre-training and evaluation”
2
LLM-assisted relevance labeling
ai_action
“We started with about 700,000 human labels, which were then used to fine-tune a GPT model and label 32 million pairs to create the final dataset”
3
Stage 1: product encoder pretraining
ai_action
“In Stage 1, we perform continual pretraining of the product encoders on 400,000 raw product image/title pairs from our catalog. This helps the encoders adapt to the characteristics and patterns of the product domain”
4
Stage 2: query-product alignment
ai_action
“In Stage 2, we align the query embedding with the product embedding by minimizing a contrastive loss in the projection space of the image-text product encoder and text-only query encoder”
5
K-NN candidate retrieval
ai_action
“We leveraged the embedding of a user's query to perform a K-nearest neighbor search in the embedding space of the product to create a ranked list of potential relevant candidates for the next downstream selection, such as ranking”
6
Ad ranking model prediction
ai_action
“DoorDash models the ranking problem as a binary classification task in which the model predicts the probability of the user clicking a given candidate ad”
7
Online A/B test and full deployment
validation
“we set up an online A/B experiment to evaluate our best candidate against online traffic for about 10 days”
Reported outcome

DashCLIP outperformed all baseline models by significant gains on retrieval tasks, improved ranking metrics, increased engagement rates and revenue for sponsored product ads, and was deployed to serve 100% of traffic.

Reported metrics
Ranking and retrieval performancesignificant performance improvement across ranking and retrieval tasks
DashCLIP vs baseline modelsoutperformed all baselines by significant gains
Engagement ratesincreased engagement rates for most of the top queries and categories
Revenue from sponsored products adsdriving more revenue for sponsored products ads
Show all 5 reported metrics
ranking and retrieval performancesignificant performance improvement across ranking and retrieval tasks
DashCLIP vs baseline modelsoutperformed all baselines by significant gains
engagement ratesincreased engagement rates for most of the top queries and categories
revenue from sponsored products adsdriving more revenue for sponsored products ads
traffic served after deployment100%
Reported stack
DashCLIPBLIP-14MGPTLLMsCLIPBLIPFLAVA
Source
https://careersatdoordash.com/blog/doordash-dashclip-multimodal-models-for-generating-semantic-embeddings/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

DashCLIP outperformed all baseline models by significant gains on retrieval tasks, improved ranking metrics, increased engagement rates and revenue for sponsored product ads, and was deployed to serve 100% of traffic.

What tools did this team use?

DashCLIP, BLIP-14M, GPT, LLMs, CLIP, BLIP, FLAVA.

What results were reported?

Ranking and retrieval performance: significant performance improvement across ranking and retrieval tasks; DashCLIP vs baseline models: outperformed all baselines by significant gains; Engagement rates: increased engagement rates for most of the top queries and categories; Revenue from sponsored products ads: driving more revenue for sponsored products ads (source-reported, not independently verified).

What failed first in this deployment?

Off-the-shelf models lacked e-commerce domain specificity and frequently failed on short but specific queries; historical engagement data introduced position and selection bias into training labels.

How is this ecommerce ops AI workflow structured?

Product catalog data ingestion → LLM-assisted relevance labeling → Stage 1: product encoder pretraining → Stage 2: query-product alignment → K-NN candidate retrieval → Ad ranking model prediction → Online A/B test and full deployment.

Related ecommerce ops cases