DoorDash best practices for regression-free ML model migration in Dasher assignment
Migrating ML models from DoorDash's legacy monolith to a microservices architecture posed high regression risk because model complexity—with dozens or hundreds of interacting features—made regressions hard to prevent, detect, and correct, and different state snapshots in the new service altered model inputs and outputs.
How it works
Common implementation structure
How this type of workflow is generally built, generalized across documented cases — not tied to any one vendor's stack. Click any stage to read what happens there. Specific products that implement these stages appear in “Tools commonly seen” below.
Stage 1 · Define metrics and success thresholds
Business metrics ASAP and DAT are selected with a switchback experiment framework as the protocol to measure migration success.
Tools used
SibylPythonC++
Outcome
After corrective measures were applied and switchback experiments re-run for each affected swap, all tests passed and the migration completed successfully, maintaining Dasher assignment quality.
What failed first
During the sequential swap of inference sources, two swaps produced statistically significant degradations in ASAP and DAT metrics, with degradations worse than the secondary success criterion, requiring corrective measures before continuing.