Workflow · Production

Netflix integrates its Foundation Model into personalization applications via embeddings, subgraph, and fine-tuning approaches

The problem

Netflix's homepage is powered by several specialized models that require significant time and resources to maintain, creating a need to centralize member preference learning into one powerful foundation model.

Workflow diagram · grounded in source
1
Foundation Model monthly pre-training
ai_action
“the Foundation Model is pre-trained from scratch every month”
2
Daily fine-tuning on latest data
ai_action
“the monthly pre-trained model is fine-tuned everyday based on the latest data. The daily fine-tuning also expands the entity id space to include the newly launching titles”
3
Batch inference refreshes embeddings
ai_action
“After the daily fine-tuned model is ready, we run batch inference to refresh the profile and item embeddings and publish them to the Embedding Store”
4
Embedding stabilization across runs
validation
“it is important to map the embeddings generated everyday into the same embedding space, so that downstream models can consume pre-computed embeddings as features”
5
Embeddings published to Embedding Store
integration
“The embedding store makes producing and consuming embeddings easy by taking care of versioning and timestamping the embeddings. The Embedding Store also provides various interfaces to access the embeddings offline and online”
6
Applications consume embeddings as features
output
“Embeddings can serve as powerful features for other models or for candidate generation, helping to retrieve appealing titles for a user or facilitate title-to-title recommendations”
7
Subgraph integration runs Foundation Model inline
ai_action
“the Foundation Model decoder stack is a subgraph of the application model's full model graph and hence the name "subgraph integration". The Foundation Model subgraph processes raw user interaction sequences and outputs representations th…”
8
Fine-tuning adapts model to application objectives
ai_action
“the Foundation Model can be fine-tuned on more product specific data and then the fine-tuned model can be used to directly power the product”
Reported outcome

Three integration approaches — embeddings, subgraph, and fine-tuning — are now used in production for different use cases, with embeddings offering a low cost and high leverage entry point and subgraph enabling deeper integration to harness the full power of the Foundation Model.

Reported metrics
Time and resources to maintain individual modelssignificant time and resources
Cost and leverage of embedding approachlow cost and high leverage
Training and inference cost impact of embedding approachrelatively smaller impact on training and inference costs
Gap in practical integration guidancesignificant gap in practical guidance and research
Reported stack
Foundation ModelEmbedding Store
Source
https://netflixtechblog.medium.com/integrating-netflixs-foundation-model-into-personalization-applications-cf176b5860eb
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Three integration approaches — embeddings, subgraph, and fine-tuning — are now used in production for different use cases, with embeddings offering a low cost and high leverage entry point and subgraph enabling deeper…

What tools did this team use?

Foundation Model, Embedding Store.

What results were reported?

Time and resources to maintain individual models: significant time and resources; Cost and leverage of embedding approach: low cost and high leverage; Training and inference cost impact of embedding approach: relatively smaller impact on training and inference costs; Gap in practical integration guidance: significant gap in practical guidance and research (source-reported, not independently verified).

How is this workflow AI workflow structured?

Foundation Model monthly pre-training → Daily fine-tuning on latest data → Batch inference refreshes embeddings → Embedding stabilization across runs → Embeddings published to Embedding Store → Applications consume embeddings as features → Subgraph integration runs Foundation Model inline → Fine-tuning adapts model to application objectives.