Back office ops · Production

LinkedIn reduces GPU memory usage by 60% for LLM training with Liger-Kernel

The problem

LinkedIn's LLM training at scale suffered from two performance bottlenecks: heavy GPU memory access from frequent data transfers between slow HBM and fast SRAM, and extra time and resources consumed per training operation.

Workflow diagram · grounded in source
1
Training bottleneck identified
trigger
“they were experiencing performance bottlenecks during training, such as: Heavy GPU memory access Extra time and resources used per-operation”
2
FlashAttention memory optimization
ai_action
“Liger-Kernel is built upon a technique called FlashAttention. This can improve GPU performance by calculating things like attention scores and partial sums on the SRAM instead of the HBM.”
3
Operator fusion across GPUs
ai_action
“they further optimized GPUs by taking tasks that needed many GPUs and merging operations so tasks could run on one GPU”
4
Triton GPU kernel implementation
output
“LinkedIn wrote their own operator fusion in Triton for Liger-Kernel. But also wrote RMSNorm and other operations in Triton to take full advantage of the GPU.”
5
Distributed training deployment
integration
“Liger-Kernel will be added to the container image, which runs on a pod. Once the pod starts, Liger-Kernel code compiles then optimizes the GPU kernel before training begins.”
Reported outcome

Liger-Kernel improved multi-GPU training throughput by 20%, reduced end-to-end training time by 3x, and reduced memory usage by 60%.

Reported metrics
multi-GPU training throughput20%
End-to-end training time3x reduction
GPU memory usage60%
Reported stack
Liger-KernelFlashAttentionPyTorchtorch.compileTritonTorch Distributed ElasticAWS
Source
https://newsletter.betterstack.com/p/how-linkedin-reduced-gpu-memory-usage
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Liger-Kernel improved multi-GPU training throughput by 20%, reduced end-to-end training time by 3x, and reduced memory usage by 60%.

What tools did this team use?

Liger-Kernel, FlashAttention, PyTorch, torch.compile, Triton, Torch Distributed Elastic, AWS.

What results were reported?

multi-GPU training throughput: 20%; End-to-end training time: 3x reduction; GPU memory usage: 60% (source-reported, not independently verified).

How is this back office ops AI workflow structured?

Training bottleneck identified → FlashAttention memory optimization → Operator fusion across GPUs → Triton GPU kernel implementation → Distributed training deployment.

Related back office ops cases