customer_support · ecommerce · workflow
DoorDash 2025 Summer Interns Build In-House LLM for Never-Delivered Order Feature Extraction and RAG-Based Chatbot Service
DoorDash's never-delivered order review process was fully manual, slow, and expensive, limiting resolutions to just a few cases per day. Separately, there was no centralized platform to manage knowledge bases or a unified API to deploy chatbots across internal teams.
How it works
Common implementation structure
How this type of workflow is generally built, generalized across documented cases — not tied to any one vendor's stack. Click any stage to read what happens there. Specific products that implement these stages appear in “Tools commonly seen” below.
Stage 1 · ND order reported
A never-delivered designation occurs when a customer reports an order was not received despite the Dasher marking it as delivered.
Tools used
Meta Llama 3DistilBertForSequenceClassificationKafkaCadenceCockroachDBAppenRAGvector databaseMosaicKotlin
Outcome
The in-house fine-tuned DistilBertForSequenceClassification model achieved an F1 score of 0.8289 and accuracy of 0.9870, automating ND feature extraction and reducing investigation costs. A centralized RAG-based chatbot service now powers the Dasher-facing assistant.
What failed first
Teams previously stored article embeddings in a vector database while managing metadata separately in spreadsheets, creating an error-prone two-step lookup workflow that did not scale.
Results
Time saved0.0936 seconds
Volume0.8289
Cost replacedreduce investigation costs and speed up response times
Grounding & classification
Source type: technical build writeup
46 fields verified against source quotes.
chatbotdata extractiondocument classificationknowledge searchragchat transcriptknowledge basesupport ticketmetric backednamed customerproduction runtime claimedtools describedworkflow describedecommerceaccuracy improvementcost reductioncycle time reductionemployee productivitytime savedtechnical build writeupback office opscustomer supportautonomous resolutionextract classify routerag answering