Ecommerce ops · Production

DoorDash uses machine learning and SHAP values to discover online menu best practices and improve merchant conversion rates

The problem

Online menus lack the visual and experiential elements of physical storefronts, leaving the menu as the sole driver of customer conversion; unattractive or poorly organized menus can significantly hurt merchant sales, yet discovering which menu characteristics drive success at scale was difficult because most menu data is unstructured.

First attempt

Initial linear regression and base tree models had high error rates and feature collinearity issues; subsequent boosted models improved accuracy but created a black box explainability problem where the directional impact of individual features was unclear.

Workflow diagram · grounded in source
1
Define menu feature set
trigger
“In building the set of features for the model, we looked at each layer of a menu, from the high-level menu appearance to detailed modifiers for each item”
2
Baseline regression modeling
ai_action
“We built our initial regression models with linear regression models and base tree models to achieve a baseline error. While the results were interpretable, the error rate was quite high. Furthermore, many of the features seemed to be co…”
3
VIF pruning
validation
“we calculated VIF values to prune overly correlated variables”
4
LightGBM / XGBoost final models
ai_action
“LightGBM and XGBoost proved the most accurate, so we chose them as the final models”
5
SHAP explainability
ai_action
“we used Shapley (SHAP) values, a game theoretical approach towards model explainability. SHAP values represent the marginal contribution of each feature to the target variable”
6
Cuisine-specific model rerun
ai_action
“rerunning the models across different merchant categories led to interesting variance in top menu factors”
7
Translate findings to A/B hypotheses
output
“we were able to translate characteristics of menus into a series of hypotheses for A/B experiments”
Reported outcome

ML models identified photo coverage, item customizability, and category mix as top conversion drivers; A/B experiments showed a hefty improvement from header photos and restaurant information; cuisine-specific model reruns revealed that optimal menu design varies significantly by restaurant type.

Reported metrics
Menu performance improvement from header photos and restaurant infohefty improvement
Reported stack
machine learningregression modelsLightGBMXGBoostSHAP
Source
https://careersatdoordash.com/blog/uncovering-online-delivery-menu-best-practices-with-machine-learning/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

ML models identified photo coverage, item customizability, and category mix as top conversion drivers; A/B experiments showed a hefty improvement from header photos and restaurant information; cuisine-specific model r…

What tools did this team use?

machine learning, regression models, LightGBM, XGBoost, SHAP.

What results were reported?

Menu performance improvement from header photos and restaurant info: hefty improvement (source-reported, not independently verified).

What failed first in this deployment?

Initial linear regression and base tree models had high error rates and feature collinearity issues; subsequent boosted models improved accuracy but created a black box explainability problem where the directional imp…

How is this ecommerce ops AI workflow structured?

Define menu feature set → Baseline regression modeling → VIF pruning → LightGBM / XGBoost final models → SHAP explainability → Cuisine-specific model rerun → Translate findings to A/B hypotheses.

Related ecommerce ops cases