Quality assurance · Production

LinkedIn builds agentic workflows to accelerate Liger Kernel GPU kernel engineering

The problem

Writing optimized GPU kernels requires deep expertise that is scarce, and demand far outpaces the supply of engineers who can write them. Maintaining the Liger Kernel project at scale — creating kernels, supporting new models, and optimizing performance — requires hours of expert time per task and does not scale with the pace of model innovation.

First attempt

Early iterations of the agentic workflows generated plausible-looking code that failed convergence tests in subtle ways, such as a wrong casting mode or an incorrect stride computation.

Workflow diagram · grounded in source
1
Engineer provides input
trigger
“The input can be a PyTorch file, a GitHub URL, a code snippet, a paper reference, or even a natural language description like "ReLU squared activation function."”
2
Agent analysis and structured profiling
ai_action
“The agent reads the input (e.g. source code, a URL, a natural language description), reasons about the problem, and produces a structured profile capturing all key decisions”
3
Human reviews structured profile
human_review
“The human reviews the profile before proceeding”
4
Agent generates or modifies files
ai_action
“Using the confirmed profile and existing Liger code as reference, the agent generates or modifies the necessary files, following project-specific conventions and patterns”
5
Automated correctness and benchmark validation
validation
“The agent runs correctness checks, benchmarks, and generates reports. Hard failures block progress; soft failures are flagged for review”
6
Optimization learning accumulation
feedback_loop
“The agent reads all prior notebooks before generating the next variant, so learning accumulates across iterations”
7
Human reviews final diff
human_review
“human effort limited to reviewing the optimization profile and the final diff”
Reported outcome

Agentic workflows automated kernel creation, model integration, and optimization, producing results including a 1.9x forward and 3.2x backward speedup with 37.5% memory reduction for the ReLU² kernel, a 3.35x backward speedup for the fused_add_rms_norm kernel, and a 10x encoder step-time improvement with 64.7% GPU hours saved in LinkedIn's internal training infrastructure.

Reported metrics
Liger Kernel throughput improvement20%
Liger Kernel memory reduction60%
Liger Kernel total downloads7 million+
Liger Kernel contributors100+
Show all 14 reported metrics
Liger Kernel throughput improvement20%
Liger Kernel memory reduction60%
Liger Kernel total downloads7 million+
Liger Kernel contributors100+
ReLU² kernel forward speedup vs PyTorch1.9x
ReLU² kernel backward speedup vs PyTorch3.2x
ReLU² kernel memory reduction vs PyTorch37.5%
fused_add_rms_norm backward speedup at hidden dim 163843.35x
fused_add_rms_norm full-pass speedup59%
Encoder step time improvement (internal training)10x
Average training step time improvement3x
GPU hours saved on end-to-end training job64.7%
GPU occupancy before optimization (fused_add_rms_norm backward)12.5%
Guardrail: max non-target metric regression allowed5%
Reported stack
Liger Kernelliger-kernel-devliger-autopatchliger-kernel-perfTritonPyTorchHuggingFace TransformersTRLLLaMa-FactoryFlash AttentionPyTorch FSDPDeepSpeedNVIDIA NCUTorchDynamotorch.compiletorch fx
Source
https://www.linkedin.com/blog/engineering/ai/ai-helping-build-better-ai-how-agents-accelerate-liger-kernel-engineering
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Agentic workflows automated kernel creation, model integration, and optimization, producing results including a 1.9x forward and 3.2x backward speedup with 37.5% memory reduction for the ReLU² kernel, a 3.35x backward…

What tools did this team use?

Liger Kernel, liger-kernel-dev, liger-autopatch, liger-kernel-perf, Triton, PyTorch, HuggingFace Transformers, TRL, LLaMa-Factory, Flash Attention.

What results were reported?

Liger Kernel throughput improvement: 20%; Liger Kernel memory reduction: 60%; Liger Kernel total downloads: 7 million+; Liger Kernel contributors: 100+ (source-reported, not independently verified).

What failed first in this deployment?

Early iterations of the agentic workflows generated plausible-looking code that failed convergence tests in subtle ways, such as a wrong casting mode or an incorrect stride computation.

How is this quality assurance AI workflow structured?

Engineer provides input → Agent analysis and structured profiling → Human reviews structured profile → Agent generates or modifies files → Automated correctness and benchmark validation → Optimization learning accumulation → Human reviews final diff.

Related quality assurance cases