Workflow · Production

Aurora builds centralized ML orchestration layer to accelerate autonomous vehicle Data Engine

The problem

Aurora's ML model development workflow was highly manual and fragmented — going from new data to a production model required significant manual effort, parallel experimentation was labor-intensive, there was no unified interface for debugging, and bottlenecks at any stage delayed continuous deployment of new models to vehicles.

Workflow diagram · grounded in source
1
Identify data requirements
trigger
“The Data Engine lifecycle starts with identifying the type of data required to support or improve an AV capability”
2
Mine and label training data
integration
“iteratively mining and labeling the data to turn it into a usable dataset for ML model training”
3
Distributed model training
ai_action
“Launches distributed model training on Sagemaker”
4
Sub-system and system evaluation
validation
“it then goes through a number of sub-system and system-level evaluations”
5
Metrics fed to Data Engine
feedback_loop
“those results are fed back to the Data Engine for the next iteration”
6
Automated CI/CD deployment
output
“automated runs of end-to-end deployment workflows to create datasets and train and deploy models on newly labeled data”
Reported outcome

Aurora drastically cut down the time spent on production and deployment of models on new data, reduced manual effort during experimentation, and sped up model development workflows — the majority of autonomy model developers now use the centralized ML orchestration system.

Reported metrics
Dataset and model delivery cadencewithin two weeks
Time spent on model production and deploymentdrastically cut down
Manual effort during experimentationreduced
Model development workflow speedsped up
Reported stack
Kubeflow PipelinesSagemakerEKSTerraformBuildkiteBazelTensorboardBatch APISlackGitHub
Source
https://mlops.community/blog/auroras-data-engine-how-we-accelerate-machine-learning-model-workflows
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Aurora drastically cut down the time spent on production and deployment of models on new data, reduced manual effort during experimentation, and sped up model development workflows — the majority of autonomy model dev…

What tools did this team use?

Kubeflow Pipelines, Sagemaker, EKS, Terraform, Buildkite, Bazel, Tensorboard, Batch API, Slack, GitHub.

What results were reported?

Dataset and model delivery cadence: within two weeks; Time spent on model production and deployment: drastically cut down; Manual effort during experimentation: reduced; Model development workflow speed: sped up (source-reported, not independently verified).

How is this workflow AI workflow structured?

Identify data requirements → Mine and label training data → Distributed model training → Sub-system and system evaluation → Metrics fed to Data Engine → Automated CI/CD deployment.