Medical records processing · Production

Accelerating DenseNet-121 medical imaging inference with GPU-native pipeline on MedNIST

The problem

Modern medical imaging workflows must process thousands of high-resolution scans rapidly, but CPU-based pipelines are slow and inefficient — cores run at low utilization while I/O stalls and host-to-device data copies add latency.

Workflow diagram · grounded in source
1
Data loading and labeling
trigger
“Read image files and assign class labels. We collected ~64K MedNIST JPEGs (6 classes) and split them into train/validation.”
2
GPU-accelerated preprocessing
integration
“we substitute CPU-based tools (Pillow/OpenCV, pandas, etc.) with NVIDIA GPU-accelerated libraries (DALI, cuDF, etc.) to remove bottlenecks and boost throughput”
3
Batching and GPU transfer
integration
“GPU path (DALI): On-GPU decoding and batching with overlap between I/O and inference. CPU path (PyTorch DataLoader): Host batching with explicit .to(device) copies, adding latency.”
4
DenseNet-121 inference
ai_action
“We run DenseNet-121 (2D, 1 input channel, 6 classes) in eval() mode with FP16 enabled to exploit GPU tensor cores”
5
Confidence score extraction
output
“Logits are passed through softmax/top-k to extract top-1 predictions and confidence scores”
6
Performance monitoring
feedback_loop
“A custom profiler logs per-batch timings plus GPU utilization, memory, and power at ~10 Hz. From these metrics, we derive latency, throughput, and energy efficiency.”
Reported outcome

The GPU-native pipeline achieved 3.3× higher throughput and 2.9× lower batch time versus CPU, with 88% GPU utilization and roughly 40% better energy efficiency measured in images per joule.

Reported metrics
GPU throughput gain vs CPU~3.3×
GPU batch time reduction vs CPU~2.9×
GPU batch time per 32 images~0.109 s per batch of 32
CPU batch time per 32 images~0.315 s
Show all 13 reported metrics
GPU throughput gain vs CPU~3.3×
GPU batch time reduction vs CPU~2.9×
GPU batch time per 32 images~0.109 s per batch of 32
CPU batch time per 32 images~0.315 s
GPU compute utilization~88%
CPU utilization~62%
GPU throughput (images per second)103 img/s
CPU throughput (images per second)31 img/s
GPU energy efficiency improvementroughly 40%
GPU VRAM usage share of total20%
HeadCT classification confidence96.3%
CXR classification confidence94.7%
AbdomenCT classification confidence95.1%
Reported stack
DenseNet-121DALIcuDFNVIDIA Tesla T4PyTorch DataLoaderPillowOpenCVpandasFP16NVMLPyTorchcuDNNcuBLAS
Source
https://mlops.community/blog/accelerating-densenet-121-inference-nvidia
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The GPU-native pipeline achieved 3.3× higher throughput and 2.9× lower batch time versus CPU, with 88% GPU utilization and roughly 40% better energy efficiency measured in images per joule.

What tools did this team use?

DenseNet-121, DALI, cuDF, NVIDIA Tesla T4, PyTorch DataLoader, Pillow, OpenCV, pandas, FP16, NVML.

What results were reported?

GPU throughput gain vs CPU: ~3.3×; GPU batch time reduction vs CPU: ~2.9×; GPU batch time per 32 images: ~0.109 s per batch of 32; CPU batch time per 32 images: ~0.315 s (source-reported, not independently verified).

How is this medical records processing AI workflow structured?

Data loading and labeling → GPU-accelerated preprocessing → Batching and GPU transfer → DenseNet-121 inference → Confidence score extraction → Performance monitoring.

Related medical records processing cases