Back office ops · Production

Liger-Kernel: LinkedIn's open-source Triton kernel library improves LLM training throughput by 20% and cuts memory usage by 60%

The problem

Training LLMs on GPUs is slowed by two key bottlenecks: extensive GPU memory I/O overhead between slow HBM and fast SRAM for every kernel launched, and per-operation overhead from eager-execution frameworks where operations run synchronously line-by-line and output activations must be stored in memory for the backward pass.

Workflow diagram · grounded in source
1
Training job submitted via platform
trigger
“Users' tasks are scheduled by Flyte onto Kubernetes, which then efficiently allocates GPUs to the tasks.”
2
Liger-Kernel applied to training code
integration
“The simplest way to leverage Liger kernels is through the AutoLigerKernelForCausalLM class. This requires no model-specific patching API imports. If the model type is supported, the modeling code will be automatically patched by Liger.”
3
Operator fusion reduces kernel overhead
output
“combining several standalone GPU kernels into one, to avoid the per-operation time and memory overhead in step-by-step execution”
4
Distributed training execution
integration
“users can use FSDP, DeepSpeed, or DDP to scale up their models, efficiently running in multi-GPU and multi-node environments”
Reported outcome

Liger-Kernel improves training throughput by 20% and reduces memory usage by 60% with a single line of code, and LinkedIn observed a 3X reduction in end-to-end training time for an in-house model at ~70B scale, with 10%–20% throughput gains at ~100B and ~10B scale.

Reported metrics
Training throughput improvement20%
Memory usage reduction60%
end-to-end training time reduction for ~70B scale model3X reduction
throughput gain for ~100B and ~10B scale LLMs10% ~ 20%
Show all 11 reported metrics
training throughput improvement20%
memory usage reduction60%
end-to-end training time reduction for ~70B scale model3X reduction
throughput gain for ~100B and ~10B scale LLMs10% ~ 20%
individual kernel execution time or memory reductionat least 30%
GitHub stars3,000+
downloads200k+
AMD throughput improvement (Embedding LLM / Hot Aisle Inc)26%
AMD memory reduction (Embedding LLM / Hot Aisle Inc)60%
JSD kernel speed improvement (128k vocab size)1.5x speed
JSD kernel peak memory reduction (128k vocab size)50%
Reported stack
Liger-KernelTritonAxolotlLLaMa-FactorySFTTrainerHugging Face TrainerSWIFTPyTorch FSDPMicrosoft DeepSpeedFlash AttentionFlyteKubernetes
Source
https://www.linkedin.com/blog/engineering/open-source/liger-kernel-open-source-ecosystem-for-efficient-llm-training
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Liger-Kernel improves training throughput by 20% and reduces memory usage by 60% with a single line of code, and LinkedIn observed a 3X reduction in end-to-end training time for an in-house model at ~70B scale, with 1…

What tools did this team use?

Liger-Kernel, Triton, Axolotl, LLaMa-Factory, SFTTrainer, Hugging Face Trainer, SWIFT, PyTorch FSDP, Microsoft DeepSpeed, Flash Attention.

What results were reported?

Training throughput improvement: 20%; Memory usage reduction: 60%; end-to-end training time reduction for ~70B scale model: 3X reduction; throughput gain for ~100B and ~10B scale LLMs: 10% ~ 20% (source-reported, not independently verified).

How is this back office ops AI workflow structured?

Training job submitted via platform → Liger-Kernel applied to training code → Operator fusion reduces kernel overhead → Distributed training execution.

Related back office ops cases