Back office ops · Production

Pinterest Feature Trimmer reduces root-leaf ML serving network bandwidth and saves over $4M annually

The problem

Pinterest's root-leaf ML serving architecture passed the full union of ML features from root to every leaf partition regardless of which features each model actually needed, creating a network bandwidth bottleneck that forced infrastructure scaling based on network utilization rather than compute and left GPU resources underutilized.

First attempt

Enabling lz4 compression in fbthrift reduced root-leaf network usage by 20% but at the cost of 5% more CPU and a 5ms (~10%) p90 latency increase, and did not address the underlying problem of transmitting unused features.

Workflow diagram · grounded in source
1
Score request from client service
Trigger
Client service sends a score request to the online ML serving root to have candidate Pins scored by ML models for relevancy.
▾ source quote
“client service sends a score request to the online ML serving system to have the Pin scored by a bunch of ML models, each of which scores an aspect of "relevancy"”
2
Root fetches features from store
Integration
The root retrieves the union of required ML features from the feature store and holds them in an in-memory cache shared across leaf partitions.
▾ source quote
“Its responsibilities include retrieving necessary features from the feature store, performing required preprocessing”
3
Feature Trimmer applies per-model allowlist
Validation
Feature Trimmer consults each model's signature and strips all features not in that model's allowlist before fan-out.
▾ source quote
“we only keep the needed features in the fan-out request to leaf partitions. This allowlist approach, compared to a blocklist where we keep features not in the list, does not carry the burden of tracking all the features that might …”
4
Root fans out trimmed features
Routing
The root distributes the trimmed, model-specific feature set to the relevant leaf partitions.
▾ source quote
“distributing (fanning out) the scoring requests to the various leaf partitions”
5
Leaf ML model inference
Ai action
Leaf partitions run ML model inference on GPU machines to produce relevance scores for candidate Pins.
▾ source quote
“This is where the actual model inference takes place, typically utilizing GPU machines”
6
Model signature sync via staged rollout
Integration
Model signatures are packaged into a bundle-level artifact and deployed to root hosts through the same canary-to-production staged pipeline as model updates.
▾ source quote
“integrates the feature trimmer into the existing model deployment system and ensures that the "root's trimming view of the world" is updated using the same guardrails and rollback mechanics as other model changes”
Reported outcome

Feature Trimmer saved over $4M in annual infrastructure costs at Pinterest, enabled a 27% Ads root cluster downsizing, reduced the Homefeed root cluster fleet by 33%, achieved roughly 45% and 65% egress drops for Search and Notification clusters, and improved Related Pins p99 latency by about 25–30%.

Reported metrics
Lz4 compression network usage reduction20%
lz4 CPU overhead5%
Lz4 p90 latency overhead5ms (~10%) p90 latency increase
Projected root-leaf bandwidth reduction from feature trimming~50%
Show all 14 reported metrics
lz4 compression network usage reduction20%
lz4 CPU overhead5%
lz4 p90 latency overhead5ms (~10%) p90 latency increase
projected root-leaf bandwidth reduction from feature trimming~50%
Ads root cluster size reduction27%
Ads GPU capacity finetunedroughly 5% of the total GPU capacity at the time
revenue increase0.17%
Search egress network throughput dropapproximately 45%
Notification egress network throughput dropapproximately 65%
Search and Notification cost reduction≥30%
annual infrastructure savings from rightsizing Search and Notification$0.98M
total annual infrastructure cost savingsover $4M
Homefeed root cluster fleet size reduction33%
Related Pins p99 latency reduction~25–30%
Reported stack
fbthriftlz4TorchScriptPyTorchGFlagsAWS
◆ Does this fit your context?

Compare to your context

Tell us your scale, team, and constraints. We'll show what changes at your size, what fails at your scale, and whether this case is a fit, needs adaptation, or won't scale to you. Free demo, no signup.

Compare to your context →
~30 seconds · free
Source
https://medium.com/pinterest-engineering/optimizing-ml-workload-network-efficiency-part-i-feature-trimmer-ae20beb08d69
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Feature Trimmer saved over $4M in annual infrastructure costs at Pinterest, enabled a 27% Ads root cluster downsizing, reduced the Homefeed root cluster fleet by 33%, achieved roughly 45% and 65% egress drops for Sear…

What tools did this team use?

fbthrift, lz4, TorchScript, PyTorch, GFlags, AWS.

What results were reported?

Lz4 compression network usage reduction: 20%; lz4 CPU overhead: 5%; Lz4 p90 latency overhead: 5ms (~10%) p90 latency increase; Projected root-leaf bandwidth reduction from feature trimming: ~50% (source-reported, not independently verified).

What failed first in this deployment?

Enabling lz4 compression in fbthrift reduced root-leaf network usage by 20% but at the cost of 5% more CPU and a 5ms (~10%) p90 latency increase, and did not address the underlying problem of transmitting unused featu…

How is this back office ops AI workflow structured?

Score request from client service → Root fetches features from store → Feature Trimmer applies per-model allowlist → Root fans out trimmed features → Leaf ML model inference → Model signature sync via staged rollout.

Related back office ops cases

WHAT TO DO WITH THIS

Now compare it to your context

This case is one data point. Whether its pattern fits you depends on your volumes, your stack, and your exception load — that comparison is the step no case study can do for you.

Browse the Back office ops cases →