Workflow · Production

Netflix Foundation Model for Personalized Recommendation: A Unified LLM-Inspired Architecture

The problem

Netflix's recommender system comprised many independently trained specialized ML models that were costly to maintain and made it difficult to transfer innovations between models, while most were confined to brief temporal windows due to serving latency and training cost constraints.

Workflow diagram · grounded in source
1
User interaction tokenization
ai_action
“Tokenization helps define what constitutes a meaningful "token" in a sequence. Drawing an analogy to Byte Pair Encoding (BPE) in NLP, we can think of tokenization as merging adjacent actions to form new, higher-level tokens.”
2
Semi-supervised next-token prediction training
ai_action
“our default approach employs the autoregressive next-token prediction objective, similar to GPT. This strategy effectively leverages the vast scale of unlabeled user interaction data.”
3
Cold-start metadata embedding initialization
ai_action
“To create the final title embedding, we combine this metadata-based embedding with a fully-learnable ID-based embedding using a mixing layer. Instead of simply summing these embeddings, we use an attention mechanism based on the "age" of…”
4
Embedding generation and storage
output
“The model generates valuable embeddings for members and entities like videos, games, and genres. These embeddings are calculated in batch jobs and stored for use in both offline and online applications.”
5
Downstream embedding distribution
integration
“They can serve as features in other models or be used for candidate generation, such as retrieving appealing titles for a user. High-quality title embeddings also support title-to-title recommendations.”
6
Application-specific fine-tuning
ai_action
“Users can integrate the full model or subgraphs into their own models, fine-tuning them with less data and computational power. This approach achieves performance comparable to previous models”
Reported outcome

The foundation model enables downstream applications to use shared embeddings and fine-tune with less data and computational power, achieving performance comparable to previous models, with promising results from downstream integrations and consistent improvements from scaling.

Reported metrics
Downstream integration resultspromising results from downstream integrations
Fine-tuned model performance vs previous modelsperformance comparable to previous models
Model performance at scaleconsistent improvements observed as we increase data and model size
Reported stack
transformer modelsKV cachingsparse attention
Source
https://netflixtechblog.medium.com/foundation-model-for-personalized-recommendation-1a0bd8e02d39
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The foundation model enables downstream applications to use shared embeddings and fine-tune with less data and computational power, achieving performance comparable to previous models, with promising results from down…

What tools did this team use?

transformer models, KV caching, sparse attention.

What results were reported?

Downstream integration results: promising results from downstream integrations; Fine-tuned model performance vs previous models: performance comparable to previous models; Model performance at scale: consistent improvements observed as we increase data and model size (source-reported, not independently verified).

How is this workflow AI workflow structured?

User interaction tokenization → Semi-supervised next-token prediction training → Cold-start metadata embedding initialization → Embedding generation and storage → Downstream embedding distribution → Application-specific fine-tuning.