Back office ops · Production

Pinterest Feature Trimmer reduces root-leaf ML serving network bandwidth and saves over $4M annually

The problem

Pinterest's root-leaf ML serving architecture passed the full union of ML features from root to every leaf partition regardless of which features each model actually needed, creating a network bandwidth bottleneck that forced infrastructure scaling based on network utilization rather than compute and left GPU resources underutilized.

First attempt

Enabling lz4 compression in fbthrift reduced root-leaf network usage by 20% but at the cost of 5% more CPU and a 5ms (~10%) p90 latency increase, and did not address the underlying problem of transmitting unused features.

Workflow diagram · grounded in source
1
Score request from client service
trigger
“client service sends a score request to the online ML serving system to have the Pin scored by a bunch of ML models, each of which scores an aspect of "relevancy"”
2
Root fetches features from store
integration
“Its responsibilities include retrieving necessary features from the feature store, performing required preprocessing”
3
Feature Trimmer applies per-model allowlist
validation
“we only keep the needed features in the fan-out request to leaf partitions. This allowlist approach, compared to a blocklist where we keep features not in the list, does not carry the burden of tracking all the features that might be in …”
4
Root fans out trimmed features
routing
“distributing (fanning out) the scoring requests to the various leaf partitions”
5
Leaf ML model inference
ai_action
“This is where the actual model inference takes place, typically utilizing GPU machines”
6
Model signature sync via staged rollout
integration
“integrates the feature trimmer into the existing model deployment system and ensures that the "root's trimming view of the world" is updated using the same guardrails and rollback mechanics as other model changes”
Reported outcome

Feature Trimmer saved over $4M in annual infrastructure costs at Pinterest, enabled a 27% Ads root cluster downsizing, reduced the Homefeed root cluster fleet by 33%, achieved roughly 45% and 65% egress drops for Search and Notification clusters, and improved Related Pins p99 latency by about 25–30%.

Reported metrics
Lz4 compression network usage reduction20%
lz4 CPU overhead5%
Lz4 p90 latency overhead5ms (~10%) p90 latency increase
Projected root-leaf bandwidth reduction from feature trimming~50%
Show all 14 reported metrics
lz4 compression network usage reduction20%
lz4 CPU overhead5%
lz4 p90 latency overhead5ms (~10%) p90 latency increase
projected root-leaf bandwidth reduction from feature trimming~50%
Ads root cluster size reduction27%
Ads GPU capacity finetunedroughly 5% of the total GPU capacity at the time
revenue increase0.17%
Search egress network throughput dropapproximately 45%
Notification egress network throughput dropapproximately 65%
Search and Notification cost reduction≥30%
annual infrastructure savings from rightsizing Search and Notification$0.98M
total annual infrastructure cost savingsover $4M
Homefeed root cluster fleet size reduction33%
Related Pins p99 latency reduction~25–30%
Reported stack
fbthriftlz4TorchScriptPyTorchGFlagsAWS
Source
https://medium.com/pinterest-engineering/optimizing-ml-workload-network-efficiency-part-i-feature-trimmer-ae20beb08d69
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Feature Trimmer saved over $4M in annual infrastructure costs at Pinterest, enabled a 27% Ads root cluster downsizing, reduced the Homefeed root cluster fleet by 33%, achieved roughly 45% and 65% egress drops for Sear…

What tools did this team use?

fbthrift, lz4, TorchScript, PyTorch, GFlags, AWS.

What results were reported?

Lz4 compression network usage reduction: 20%; lz4 CPU overhead: 5%; Lz4 p90 latency overhead: 5ms (~10%) p90 latency increase; Projected root-leaf bandwidth reduction from feature trimming: ~50% (source-reported, not independently verified).

What failed first in this deployment?

Enabling lz4 compression in fbthrift reduced root-leaf network usage by 20% but at the cost of 5% more CPU and a 5ms (~10%) p90 latency increase, and did not address the underlying problem of transmitting unused featu…

How is this back office ops AI workflow structured?

Score request from client service → Root fetches features from store → Feature Trimmer applies per-model allowlist → Root fans out trimmed features → Leaf ML model inference → Model signature sync via staged rollout.

Related back office ops cases