Democratizing Machine Learning at Netflix: Building the Model Lifecycle Graph
As Netflix's ML investments scaled across business domains, models became black boxes with no discovery infrastructure — practitioners had to traverse fragmented, siloed tools to answer basic questions about lineage, ownership, and impact, and cross-domain reuse of ML assets was extraordinarily difficult.
The siloed tooling left each system unaware of the others — the model registry did not know which A/B tests used its models, the pipeline orchestrator was unaware of downstream model dependencies, and practitioners had no way to answer cross-domain impact or lineage questions.
Netflix built MDS (Metadata Service) with the Model Lifecycle Graph, enabling every ML practitioner to discover, understand, and reuse ML assets across all domains through the AIP Portal, replacing multi-system manual investigation with single graph queries.
Frequently asked questions
What did this team achieve with this AI workflow?
Netflix built MDS (Metadata Service) with the Model Lifecycle Graph, enabling every ML practitioner to discover, understand, and reuse ML assets across all domains through the AIP Portal, replacing multi-system manual…
What tools did this team use?
Kafka, AWS SNS/SQS, Datomic, Elasticsearch.
What results were reported?
Relationship materialization delay: typically minutes rather than seconds; Cross-domain query simplification: single query (source-reported, not independently verified).
What failed first in this deployment?
The siloed tooling left each system unaware of the others — the model registry did not know which A/B tests used its models, the pipeline orchestrator was unaware of downstream model dependencies, and practitioners ha…
How is this back office ops AI workflow structured?
Source system event ingestion → Entity hydration from source APIs → Normalization to unified entity model → Persist to Datomic and index in Elasticsearch → Background relationship enrichment → AIP Portal graph exploration.