Medical records processing · Production

Accelerating DenseNet-121 medical imaging inference with GPU-native pipeline on MedNIST

The problem

Modern medical imaging workflows must process thousands of high-resolution scans rapidly, but CPU-based pipelines are slow and inefficient — cores run at low utilization while I/O stalls and host-to-device data copies add latency.

Workflow diagram · grounded in source
1
Data loading and labeling
Trigger
Image files are read and class labels are assigned from approximately 64K MedNIST JPEGs across 6 classes.
▾ source quote
“Read image files and assign class labels. We collected ~64K MedNIST JPEGs (6 classes) and split them into train/validation.”
2
GPU-accelerated preprocessing
Integration
CPU-based tools (Pillow/OpenCV, pandas) are substituted with NVIDIA GPU-accelerated libraries (DALI, cuDF) to remove bottlenecks and boost throughput.
▾ source quote
“we substitute CPU-based tools (Pillow/OpenCV, pandas, etc.) with NVIDIA GPU-accelerated libraries (DALI, cuDF, etc.) to remove bottlenecks and boost throughput”
3
Batching and GPU transfer
Integration
The GPU path uses DALI for on-GPU decoding and batching with overlap between I/O and inference.
▾ source quote
“GPU path (DALI): On-GPU decoding and batching with overlap between I/O and inference. CPU path (PyTorch DataLoader): Host batching with explicit .to(device) copies, adding latency.”
4
DenseNet-121 inference
Ai action
DenseNet-121 runs in eval mode with FP16 enabled to exploit GPU tensor cores for 6-class medical image classification.
▾ source quote
“We run DenseNet-121 (2D, 1 input channel, 6 classes) in eval() mode with FP16 enabled to exploit GPU tensor cores”
5
Confidence score extraction
Output
Logits are passed through softmax/top-k to extract top-1 predictions and confidence scores.
▾ source quote
“Logits are passed through softmax/top-k to extract top-1 predictions and confidence scores”
6
Performance monitoring
Feedback loop
A custom profiler logs per-batch timings plus GPU utilization, memory, and power at approximately 10 Hz to derive latency, throughput, and energy efficiency.
▾ source quote
“A custom profiler logs per-batch timings plus GPU utilization, memory, and power at ~10 Hz. From these metrics, we derive latency, throughput, and energy efficiency.”
Reported outcome

The GPU-native pipeline achieved 3.3× higher throughput and 2.9× lower batch time versus CPU, with 88% GPU utilization and roughly 40% better energy efficiency measured in images per joule.

Reported metrics
GPU throughput gain vs CPU~3.3×
GPU batch time reduction vs CPU~2.9×
GPU batch time per 32 images~0.109 s per batch of 32
CPU batch time per 32 images~0.315 s
Show all 13 reported metrics
GPU throughput gain vs CPU~3.3×
GPU batch time reduction vs CPU~2.9×
GPU batch time per 32 images~0.109 s per batch of 32
CPU batch time per 32 images~0.315 s
GPU compute utilization~88%
CPU utilization~62%
GPU throughput (images per second)103 img/s
CPU throughput (images per second)31 img/s
GPU energy efficiency improvementroughly 40%
GPU VRAM usage share of total20%
HeadCT classification confidence96.3%
CXR classification confidence94.7%
AbdomenCT classification confidence95.1%
Reported stack
DenseNet-121DALIcuDFNVIDIA Tesla T4PyTorch DataLoaderPillowOpenCVpandasFP16NVMLPyTorchcuDNNcuBLAS
◆ Does this fit your context?

Compare to your context

Tell us your scale, team, and constraints. We'll show what changes at your size, what fails at your scale, and whether this case is a fit, needs adaptation, or won't scale to you. Free demo, no signup.

Compare to your context →
~30 seconds · free
Source
https://mlops.community/blog/accelerating-densenet-121-inference-nvidia
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The GPU-native pipeline achieved 3.3× higher throughput and 2.9× lower batch time versus CPU, with 88% GPU utilization and roughly 40% better energy efficiency measured in images per joule.

What tools did this team use?

DenseNet-121, DALI, cuDF, NVIDIA Tesla T4, PyTorch DataLoader, Pillow, OpenCV, pandas, FP16, NVML.

What results were reported?

GPU throughput gain vs CPU: ~3.3×; GPU batch time reduction vs CPU: ~2.9×; GPU batch time per 32 images: ~0.109 s per batch of 32; CPU batch time per 32 images: ~0.315 s (source-reported, not independently verified).

How is this medical records processing AI workflow structured?

Data loading and labeling → GPU-accelerated preprocessing → Batching and GPU transfer → DenseNet-121 inference → Confidence score extraction → Performance monitoring.

Related medical records processing cases

WHAT TO DO WITH THIS

Now compare it to your context

This case is one data point. Whether its pattern fits you depends on your volumes, your stack, and your exception load — that comparison is the step no case study can do for you.

Browse the Medical records processing cases →