Back office ops · Production

DoorDash builds a gigascale ML feature store with Redis hashes, xxHash, and Snappy compression to triple cluster capacity

The problem

DoorDash's existing Redis-based feature store had significant inefficiencies and was approaching capacity limits while needing to serve billions of feature records with millions of lookups per second for ML model inference under low-latency constraints.

First attempt

The existing Redis feature store stored features as a flat list of key-value pairs, which was memory-inefficient and compute-intensive, and the production cluster was running close to its capacity limits.

Workflow diagram · grounded in source
1
Benchmark 5 key-value stores
validation
“We ran a full-fledged benchmark evaluation on five different key-value stores to compare their cost and performance metrics”
2
Select Redis as optimal store
routing
“Our benchmarking results indicated that Redis was the best option, so we decided to optimize our feature storage mechanism”
3
Restructure to Redis hashes
integration
“we changed our storage pattern from a flat list of key-value pairs to a Redis hash per entity”
4
xxHash feature name encoding
integration
“Using a string hash function guarantees that we will have consistent references of a feature name as integer across all systems”
5
Protocol buffer serialization
integration
“To serialize compound data types, we used bytes returned via protocol buffer format”
6
Snappy compression on integer lists
integration
“We chose Snappy for its large compression ratio and low deserialization overheads”
7
Production deployment and validation
output
“When we applied the above optimizations to production Redis clusters, we observed perfectly analogous gains, a two and half times reduction”
Reported outcome

After implementing Redis hashes, xxHash string hashing, and Snappy compression, DoorDash reduced production cluster memory from 298 GB to 112 GB per billion features, cut CPU from 208 to 72 vCPUs per 10 million reads per second, and improved Redis read latency by 40% and overall feature store latency by 15%.

Reported metrics
Redis latency decrease38%
Production cluster memory per billion features298 GB RAM to 112 GB RAM per billion features
CPU utilization per 10 million reads per second208 vCPUs to 72 vCPUs per 10 million reads-per-second
Redis read latency improvement40%
Show all 10 reported metrics
Redis latency decrease38%
production cluster memory per billion features298 GB RAM to 112 GB RAM per billion features
CPU utilization per 10 million reads per second208 vCPUs to 72 vCPUs per 10 million reads-per-second
Redis read latency improvement40%
overall feature store interface latency improvement15%
cluster memory savings from string hashing15%
production cluster capacity increasenearly three times
cost reductiontripling our cost reduction
production cluster memory reduction factora two and half times reduction
write-to-read ratio in production0.1%
Reported stack
RedisAWS ElastiCacheYCSBxxHashSnappyDocker
Source
https://careersatdoordash.com/blog/building-a-gigascale-ml-feature-store-with-redis/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

After implementing Redis hashes, xxHash string hashing, and Snappy compression, DoorDash reduced production cluster memory from 298 GB to 112 GB per billion features, cut CPU from 208 to 72 vCPUs per 10 million reads…

What tools did this team use?

Redis, AWS ElastiCache, YCSB, xxHash, Snappy, Docker.

What results were reported?

Redis latency decrease: 38%; Production cluster memory per billion features: 298 GB RAM to 112 GB RAM per billion features; CPU utilization per 10 million reads per second: 208 vCPUs to 72 vCPUs per 10 million reads-per-second; Redis read latency improvement: 40% (source-reported, not independently verified).

What failed first in this deployment?

The existing Redis feature store stored features as a flat list of key-value pairs, which was memory-inefficient and compute-intensive, and the production cluster was running close to its capacity limits.

How is this back office ops AI workflow structured?

Benchmark 5 key-value stores → Select Redis as optimal store → Restructure to Redis hashes → xxHash feature name encoding → Protocol buffer serialization → Snappy compression on integer lists → Production deployment and validation.

Related back office ops cases