Workflow · Production

How Meta animates AI-generated images at scale using latency optimization and traffic management

The problem

Meta's animate feature needed to serve billions of users with fast generation times and minimal errors while remaining resource efficient, and early testing revealed that end-to-end latency was higher than expected due to global traffic routing adding seconds to generation time.

First attempt

Global traffic routing added seconds to end-to-end generation time, and after the traffic management system was added to fix that, the retry polling system began cascading under load spikes because the router had neither delay nor backoff.

Workflow diagram · grounded in source
1
User requests animation
trigger
“Meta AI's animate feature, which lets people generate a short animation of a generated image”
2
Float16 precision conversion
ai_action
“We converted the model from float32 to float16, which speeds up the inference time for two reasons. First, the memory footprint of the model is halved.”
3
DPM-Solver sampling reduction
ai_action
“We leveraged DPM-Solver and a linear-in-log signal-to-noise time to reduce the number of sampling steps to 15.”
4
Guidance and step distillation
ai_action
“our final model required only eight solver steps, with just one forward pass through the U-Net per step. In the end, during training we distilled 32 teacher steps into eight student steps.”
5
TorchScript deployment optimization
ai_action
“By converting the model to TorchScript, we achieved many automatic optimizations. These included continuous folding, fusing multiple operations, and reducing the complexity of the computational graph.”
6
Regional traffic routing
routing
“we utilized a traffic management system that fetches the service's traffic or load data and uses that to calculate a routing table. The primary objective of the routing table is to keep as many requests as possible in the same region as …”
7
Server load limiting
validation
“we made sure that the server load—queued requests plus in-flight requests—is at most one, and that the server rejects other new requests”
8
Retry delay and backoff
validation
“we modified these retry settings to add a marginal execution delay to a percentage of jobs at scheduling time—making them available to execute gradually instead of all at once—as well as an exponential backoff”
9
Animation delivered at scale
output
“we had a deployed model that was highly efficient, functioned at scale, and could handle global traffic with high availability and a minimum failure rate”
Reported outcome

After applying all optimizations, Meta deployed the animate feature with high availability and a minimum failure rate, handling global traffic with latency at expected levels.

Reported metrics
User base scalebillions of people
Model memory footprinthalved
DPM-Solver sampling steps15
Forward passes per solver step (guidance distillation)cutting inference by a factor of three
Show all 7 reported metrics
user base scalebillions of people
model memory footprinthalved
DPM-Solver sampling steps15
forward passes per solver step (guidance distillation)cutting inference by a factor of three
student solver steps distilled from teacher steps8 from 32
end-to-end latency after traffic managementlatency dropped to roughly what we would expect
service availability and failure ratehigh availability and a minimum failure rate
Reported stack
DPM-Solverbfloat16TorchScriptPyTorchpytorch.compileU-Net
Source
https://engineering.fb.com/2024/08/14/production-engineering/how-meta-animates-ai-generated-images-at-scale/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

After applying all optimizations, Meta deployed the animate feature with high availability and a minimum failure rate, handling global traffic with latency at expected levels.

What tools did this team use?

DPM-Solver, bfloat16, TorchScript, PyTorch, pytorch.compile, U-Net.

What results were reported?

User base scale: billions of people; Model memory footprint: halved; DPM-Solver sampling steps: 15; Forward passes per solver step (guidance distillation): cutting inference by a factor of three (source-reported, not independently verified).

What failed first in this deployment?

Global traffic routing added seconds to end-to-end generation time, and after the traffic management system was added to fix that, the retry polling system began cascading under load spikes because the router had neit…

How is this workflow AI workflow structured?

User requests animation → Float16 precision conversion → DPM-Solver sampling reduction → Guidance and step distillation → TorchScript deployment optimization → Regional traffic routing → Server load limiting → Retry delay and backoff → Animation delivered at scale.