Workflow · Production

Pinterest scales recommendation system serving throughput 7x with request-level deduplication

The problem

Scaling Pinterest's recommendation models 100x created massive infrastructure pressure: storage, training, and serving costs threatened to grow proportionally, requiring deliberate efficiency techniques to remain economically viable.

First attempt

Early experiments with request-sorted data caused 1–2% regressions in offline evaluation metrics due to IID disruption in Batch Normalization, and a false negative rate that jumped to as high as 30% in retrieval training.

Workflow diagram · grounded in source
1
User opens feed
trigger
“A request is triggered when a user opens their feed, kicking off the recommendation funnel”
2
Deduplicated data stored via Iceberg
integration
“By leveraging Apache Iceberg with user ID and request ID based sorting, we achieve 10–50x storage compression on user-heavy feature columns”
3
Retrieval user tower runs once
ai_action
“we rewrite the forward pass to run the user tower on the deduplicated batch of R unique requests rather than the full batch of B user-item pairs”
4
DCAT processes user history once
ai_action
“Apply the transformer to the user's historical action sequence once per deduplicated request. The keys and values (KV) from each layer are cached.”
5
Candidate items cross-attend to cached context
ai_action
“Each candidate item performs cross-attention with the cached user history KV, reusing the deduplicated context computation”
6
Ranked recommendations served
output
“The result is a 7x increase in ranking serving throughput”
Reported outcome

Request-level deduplication delivered 10–50x storage compression, a 4x retrieval training speedup, a 2.8x ranking training speedup, and a 7x serving throughput increase, enabling deployment of a 100x larger Foundation Model without proportional cost increases.

Reported metrics
Transformer dense parameter count increase100x
Model dimension increase10x
Storage compression on user-heavy feature columns10–50x
End-to-end training speedup for retrieval4x
Show all 10 reported metrics
transformer dense parameter count increase100x
model dimension increase10x
storage compression on user-heavy feature columns10–50x
end-to-end training speedup for retrieval4x
training speedup for ranking~2.8x
data loading speedup contribution to ranking40%
DCAT cross-attention speedup gain2x
ranking serving throughput increase7x
offline metric regression from request-sorted data (initial, subsequently resolved)1–2%
false negative rate with request-sorted data (initial, subsequently resolved)~30%
Reported stack
Apache IcebergTritonFlashAttention
Source
https://medium.com/pinterest-engineering/scaling-recommendation-systems-with-request-level-deduplication-93bd514142d9
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Request-level deduplication delivered 10–50x storage compression, a 4x retrieval training speedup, a 2.8x ranking training speedup, and a 7x serving throughput increase, enabling deployment of a 100x larger Foundation…

What tools did this team use?

Apache Iceberg, Triton, FlashAttention.

What results were reported?

Transformer dense parameter count increase: 100x; Model dimension increase: 10x; Storage compression on user-heavy feature columns: 10–50x; End-to-end training speedup for retrieval: 4x (source-reported, not independently verified).

What failed first in this deployment?

Early experiments with request-sorted data caused 1–2% regressions in offline evaluation metrics due to IID disruption in Batch Normalization, and a false negative rate that jumped to as high as 30% in retrieval train…

How is this workflow AI workflow structured?

User opens feed → Deduplicated data stored via Iceberg → Retrieval user tower runs once → DCAT processes user history once → Candidate items cross-attend to cached context → Ranked recommendations served.