Quality assurance · Production

NVIDIA automates GPU attention kernel generation with DeepSeek-R1 and inference-time scaling

The problem

Creating optimized GPU attention kernels requires significant skill and time even for experienced engineers, and LLMs face challenges generating correct optimized kernel code on the first try due to hallucinations, syntax errors, and non-trivial GPU thread mapping.

First attempt

LLMs used directly for kernel generation produce hallucinated code, mix syntax from different languages or frameworks, and require iterative refinement to achieve correct and efficient thread mapping.

Workflow diagram · grounded in source
1
Manual prompt initializes workflow
trigger
“The workflow is first initialized by a manual prompt”
2
DeepSeek-R1 generates GPU kernel
ai_action
“the DeepSeek-R1 model generates the GPU code (that is, the kernel) in the first pass”
3
Verifier analyzes kernel on H100
validation
“The verifier runs on an NVIDIA H100 GPU. It analyzes the generated kernel”
4
Closed-loop feedback to model
feedback_loop
“creates new prompts that are provided as input to the DeepSeek-R1 model. This closed-loop approach makes the code generation process better by guiding it in a different way each time. The team found that by letting this process continue …”
5
Numerically correct kernel output
output
“This workflow produced numerically correct kernels for 100% of Level-1 problems and 96% of Level-2 problems”
Reported outcome

The closed-loop workflow produced numerically correct kernels for 100% of Level-1 problems and 96% of Level-2 problems, with results in some cases better than kernels developed by skilled engineers.

Reported metrics
Level-1 problem correctness rate100%
Level-2 problem correctness rate96%
Inference-time budget per problem15 minutes
Kernel quality vs skilled engineersbetter than the optimized kernels developed by skilled engineers in some cases
Reported stack
DeepSeek-R1H100KernelBench
Source
https://developer.nvidia.com/blog/automating-gpu-kernel-generation-with-deepseek-r1-and-inference-time-scaling/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The closed-loop workflow produced numerically correct kernels for 100% of Level-1 problems and 96% of Level-2 problems, with results in some cases better than kernels developed by skilled engineers.

What tools did this team use?

DeepSeek-R1, H100, KernelBench.

What results were reported?

Level-1 problem correctness rate: 100%; Level-2 problem correctness rate: 96%; Inference-time budget per problem: 15 minutes; Kernel quality vs skilled engineers: better than the optimized kernels developed by skilled engineers in some cases (source-reported, not independently verified).

What failed first in this deployment?

LLMs used directly for kernel generation produce hallucinated code, mix syntax from different languages or frameworks, and require iterative refinement to achieve correct and efficient thread mapping.

How is this quality assurance AI workflow structured?

Manual prompt initializes workflow → DeepSeek-R1 generates GPU kernel → Verifier analyzes kernel on H100 → Closed-loop feedback to model → Numerically correct kernel output.

Related quality assurance cases