Back office ops · Production

Netflix introduces Metaflow spin for notebook-like iterative ML/AI development

The problem

ML and AI development involves slow iteration cycles due to long-running data transformations, model training, and stochastic processes, and the existing Metaflow resume command restarted execution from a selected step, introducing latency rather than enabling near-instant single-step feedback.

First attempt

The existing resume command restarted execution from the selected step onward, introducing latency between iterations, unlike notebooks that allow near-instant feedback by reusing data held in memory.

Workflow diagram · grounded in source
1
Create and run flow skeleton
trigger
“we start by creating a flow from scratch and running a minimal version of it to persist test artifacts”
2
Spin individual step
validation
“it quickly executes a single Metaflow @step — with all the state carried over from the parent step”
3
Iterate on step logic
feedback_loop
“we can use spin to iterate on one step at a time, incrementally building out the flow”
4
AI agent tests with spin
ai_action
“we can see Claude using spin around the 45-second mark to test a preprocess step. The step initially fails due to a classic data science pitfall: during testing, Claude samples only a small subset of data, causing some classes to be unde…”
5
Deploy to production
integration
“it can be seamlessly deployed to production orchestrators like Maestro or Argo, and scaled up on compute platforms such as AWS Batch, Titus, Kubernetes and more”
Reported outcome

The new spin command in Metaflow 2.19 executes a single @step with state carried over from the parent step, making development as smooth as a notebook while producing a production-ready, scalable workflow; AI coding agents using spin surface and fix errors more rapidly.

Reported metrics
Development iteration experienceas smooth as developing in a notebook
AI agent error surfacing speedsurface errors faster
Reported stack
MetaflowMaestroArgoAWS BatchTitusKubernetesClaude CodeJupyterVS CodeCursor
Source
https://netflixtechblog.com/supercharging-the-ml-and-ai-development-experience-at-netflix-b2d5b95c63eb
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The new spin command in Metaflow 2.19 executes a single @step with state carried over from the parent step, making development as smooth as a notebook while producing a production-ready, scalable workflow; AI coding a…

What tools did this team use?

Metaflow, Maestro, Argo, AWS Batch, Titus, Kubernetes, Claude Code, Jupyter, VS Code, Cursor.

What results were reported?

Development iteration experience: as smooth as developing in a notebook; AI agent error surfacing speed: surface errors faster (source-reported, not independently verified).

What failed first in this deployment?

The existing resume command restarted execution from the selected step onward, introducing latency between iterations, unlike notebooks that allow near-instant feedback by reusing data held in memory.

How is this back office ops AI workflow structured?

Create and run flow skeleton → Spin individual step → Iterate on step logic → AI agent tests with spin → Deploy to production.

Related back office ops cases