Quality assurance · Production

DoorDash builds ML Workbench to streamline internal machine learning operations and feature management

The problem

DoorDash ML practitioners lacked a centralized hub for ML lifecycle tasks, relying on multi-step CLI processes and tedious local workflows for routine tasks like checking feature upload status and validating feature values in production.

First attempt

The earlier ML Portal was built on Python Flask and HTML and suffered from poor information architecture and a difficult UI; users could not discover available capabilities and engineers could not use it to accelerate their own work.

Workflow diagram · grounded in source
1
Daily feature freshness check
trigger
“Model owners often perform daily checks to ensure feature freshness.”
2
Feature store integration
integration
“By enabling MLW to integrate with the feature stores, we let users directly query the production data via a simple user interface.”
3
Feature upload status check
validation
“enabling MLW to interact with the feature upload service and its tables, ensuring direct interaction with the feature service from the UI”
4
Feature value spot check
validation
“I can send it to my xfn to check features values (for pick score) and they can validate that the features are correct & make sense”
5
Model prediction testing
ai_action
“Ability to test model predictions”
Reported outcome

ML Workbench greatly reduced operational overhead for feature store queries and feature upload status checks, with data scientists reporting time savings and improved cross-functional collaboration on feature validation.

Reported metrics
ML practitioner operational overheadgreatly reduced
Feature upload status check processmuch easier and quicker
ML engineer timesaving me time
Reported stack
ML PortalReactPrismfabricatorDagitRedisExperimentation PlatformMetrics Platform
Source
https://careersatdoordash.com/blog/transforming-mlops-at-doordash-with-machine-learning-workbench/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

ML Workbench greatly reduced operational overhead for feature store queries and feature upload status checks, with data scientists reporting time savings and improved cross-functional collaboration on feature validation.

What tools did this team use?

ML Portal, React, Prism, fabricator, Dagit, Redis, Experimentation Platform, Metrics Platform.

What results were reported?

ML practitioner operational overhead: greatly reduced; Feature upload status check process: much easier and quicker; ML engineer time: saving me time (source-reported, not independently verified).

What failed first in this deployment?

The earlier ML Portal was built on Python Flask and HTML and suffered from poor information architecture and a difficult UI; users could not discover available capabilities and engineers could not use it to accelerate…

How is this quality assurance AI workflow structured?

Daily feature freshness check → Feature store integration → Feature upload status check → Feature value spot check → Model prediction testing.

Related quality assurance cases