Back office ops · Production

Scaling ML model development with MLflow: a Python SDK for experiment tracking

The problem

Data science teams needed to track ML experiments and artifacts without restructuring existing model code to fit MLflow's context manager pattern.

First attempt

The standard MLflow context manager required data scientists to rewrite training code, and MLflow's autolog functions did not automatically save all generated model files to the MLflow server.

Workflow diagram · grounded in source

Data scientist starts tracking

trigger

“if tracking: # if you decide to track: experiment_tracking.start_trianing_job(input_parameters)”

Tracking URI and experiment configured

integration

“runner.set_tracking_uri() print("Setting up experiment id") runner.set_experiment_id()”

Autolog captures training metrics

output

“mlflow.autolog() mlflow.xgboost.autolog(log_input_examples=True) mlflow.sklearn.autolog(log_models=True, log_input_examples=True, log_model_signatures=True,)”

Remaining artifacts uploaded

output

“One of the problems in avoiding the context manager is that not all the generated model files will be automatically saved to the MLflow server.”

Reported outcome

An MLflow SDK wrapper enables data scientists to add experiment tracking to existing models without restructuring code, with autolog capturing parameters and metrics and a fallback function uploading any remaining artifacts.

Reported stack

MLflowMlflowClientPlotly

Source

https://mlops.community/blog/scaling-ml-model-development-with-mlflow

Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

An MLflow SDK wrapper enables data scientists to add experiment tracking to existing models without restructuring code, with autolog capturing parameters and metrics and a fallback function uploading any remaining art…

What tools did this team use?

MLflow, MlflowClient, Plotly.

What failed first in this deployment?