Logistics ops · Production

DoorDash upgrades store-closure heuristic with ML to save thousands of canceled orders weekly

The problem

DoorDash lacked accurate real-time merchant operational status, causing Dashers to arrive at physically closed stores. Human review of thousands of daily 'store closed' reports was too costly and slow to scale.

First attempt

A heuristic rule-based approach to validating DRSC reports was only moderately accurate, and errors in passing inaccurate reports were costly to merchants and painful for customers.

Workflow diagram · grounded in source
1
Dasher reports store closed
trigger
“When Dashers find themselves unable to pick up an order at a store location that appears closed, they are prompted to take a picture of the store to kick off the reporting process”
2
Image classifier scores storefront photo
ai_action
“By converting images into a summary signal, such as the probability that a "store is closed" or a "store is open" we can process and use hundreds of thousands of images quickly. We accomplished this by training an image classifier using …”
3
LightGBM computes closure probability
ai_action
“a single LightGBM model can combine historical and image information to compute a probability that a store is closed”
4
Threshold-based action routing
routing
“low probability of a store being closed leads to unassign the order and find a new Dasher; intermediate probabilities would lead us to cancel the order; high probabilities would lead us to both cancel the order and pause the store”
5
Merchant confirmation contact
human_review
“we need to contact the merchant to confirm that the store is indeed closed, so we can pause it on the platform. If the merchant confirms the closure or is unresponsive, we will pause the store for a set period of time”
6
Order outcome and store status update
output
“we could either cancel the delivery and reimburse the customer, or alternatively when we have reason to doubt the report's accuracy, we could reassign the order to a new Dasher to re-attempt the pickup”
Reported outcome

After deploying the ML model and validating through an A/B experiment, the improved decision-making saves thousands of deliveries from being canceled every week, producing better consumer experience, increased merchant revenue, and more Dasher earning opportunities.

Reported metrics
Canceled deliveries saved per weekthousands of deliveries from being canceled every week
Reported stack
LightGBM
Source
https://careersatdoordash.com/blog/how-doordash-upgraded-a-heuristic-with-ml-to-save-thousands-of-canceled-orders/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

After deploying the ML model and validating through an A/B experiment, the improved decision-making saves thousands of deliveries from being canceled every week, producing better consumer experience, increased merchan…

What tools did this team use?

LightGBM.

What results were reported?

Canceled deliveries saved per week: thousands of deliveries from being canceled every week (source-reported, not independently verified).

What failed first in this deployment?

A heuristic rule-based approach to validating DRSC reports was only moderately accurate, and errors in passing inaccurate reports were costly to merchants and painful for customers.

How is this logistics ops AI workflow structured?

Dasher reports store closed → Image classifier scores storefront photo → LightGBM computes closure probability → Threshold-based action routing → Merchant confirmation contact → Order outcome and store status update.

Related logistics ops cases